System and method for storing/caching, searching for, and accessing data

ABSTRACT

A system for persistently maintaining data using a network for data packets is provided. The system includes a transmission medium associated with the network, a plurality of switches operatively connected to the transmission medium. Each switch has an intelligent network controller adapted for delivering the data packets to a device operatively connected to the intelligent network controller in response to a request for the data packets from the device and further adapted for re-transmitting unexpired data packets over the network.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation application of U.S. Ser. No.13/410,787 filed Mar. 2, 2012, which is a continuation of U.S. Ser. No.10/345,766 filed Jan. 16, 2003, now U.S. Pat. No. 8,165,146 issued Apr.24, 2012, which is a conversion of and claims priority to prior U.S.Provisional Patent Applications, Ser. No. 60/366,803 filed Mar. 22,2002, entitled SYSTEM AND METHOD FOR STORING/CACHING DATA ONTRANSMISSION INFRASTRUCTURE, Ser. No. 60/349,072 filed Jan. 16, 2002entitled METHOD AND SYSTEM FOR STORING DATA ON TRANSMISSION MEDIUMS;Ser. No. 10/345,766 is also a Continuation-in-Part of U.S. patentapplication Ser. No. 09/698,793 filed Oct. 27, 2000 entitled METHOD OFTRANSMITTING DATA INCLUDING A STRUCTURED LINEAR DATABASE, now U.S. Pat.No. 6,868,419 issued Mar. 15, 2005, which claims priority to 60/162,094filed Oct. 28, 1999, 60/163,426 filed Nov. 3, 1999, and 60/220,749 filedJul. 26, 2000, all of which are entitled STRUCTURTED LINEAR DATABASESand are all herein incorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to storing/caching data directlyon transmission mediums and network transmission hardware. Moreparticularly, the present invention relates to systems and methods forsearching, accessing, querying, and performing computations of locallyor globally distributed data, stored/cached in the form of data packets,protocol data units (PDU), or protocol payloads, etc., continuouslytransmitted on a telecommunication network, and/or on a microprocessor,data bus, or electronic circuit for the life of the data packet(s)and/or data stream(s).

2. Problems in the Art

Each year it is estimated that between 1 and 2 billion gigabytes ofunique information is created, and a high percentage of this informationis created in a digital format. Of all that data, 90 percent is expectedto be stored/cached digitally. Contributing to this is the rapid growthof digitized books, magazines, videos, music, and other ‘rich content’and ‘mass access’ data. This growing volume of data is often locatedremotely, must be transported between computing devices, andstored/cached in a highly secure, accessible method. The exponentialgrowth rate of generated data is expected to outpace improvements incommunication bandwidth and storage/caching capacity in the near future.Consequently, this creates an urgent need to store/cache digitalinformation in new ways that make it accessible at high speeds on astorage/caching medium and where the medium may be exponentiallyimproved.

The access speed to digital information is ultimately controlled by theinput/output (I/O) capacity of any electronic device. I/O is thelifeblood of computing, getting relevant information into and out of theprocessor, compute device, or appliance to the end-user on a timelybasis. This has always been true, but never so much as in a networkedcomputing environment.

Many associated I/O problems impede high speed access to remotelystored/cached data. Ethernet and TCP/IP are widely accepted, butinefficient protocols, which are used to drive LANs, WANs, andultimately the Internet. The TCP/IP protocol suite has proven itself abasic foundation for communications of all kinds over essentiallyunreliable networks. But that fact alone makes it inefficient andcreates network latency issues. TCP/IP-based protocols have a complex,layered design, with many inter-layer dependencies that can easilydemand extensive processing and significant buffer memory to implement.Open Systems Interconnection (OSI) is a worldwide communicationsstandard that defines a networking framework for implementing protocolsin seven layers. Handling gigabit-class network traffic, servicinginterrupts, moving data through long code-paths, and numerouskernel-to-application context switches are all expensive operations.Together, they yield long message latencies and use up a significantpercentage of available processor power.

Another problem that impedes the high speed access to data is thevenerable Peripheral Component Interconnect (PCI) shared data bus, whichis one of the most prevalent I/O architecture for compute devices. Forexample, the bandwidth for board-level transfer, and processor to cachetransfer on a typical PC is much higher than from the PC to a peripheralnetwork device via PCI bus.

Data storage/caching centers have developed a variety of specializednetworks, such as SANs (Storage Area Networks), specialized clusterlinks, NAS (Network Attached Storage), and RAID (Redundant Array ofIndependent Disks) systems in order to improve access to local andremotely distributed data. However, as RAID, NAS, Fast and GigabitEthernet, SANs, and SCSI (Small Computer System Interface) links areusually implemented with PCI adapter cards or PCI components, all of thedata traffic on these network devices is ultimately throttled by thelow-speed I/O devices.

I/O problems are further complicated by the architecture of a typicaldata storage/caching center. For example, a data storage/caching centerfor Web applications or Business-to-Business (B2B) exchanges may havehundreds of servers all requiring shared access to terabytes of filestorage/caching. The workload is defined by server requests coming inthrough networked routers, switches, firewalls, load balancers, cachingappliances, and the like. Since file sharing by multiple servers is afundamental requirement of this environment, storage/caching is usuallyaggregated into shared storage/caching pools, accessed by the serversusing a file access protocol such as Network File System (NFS). Theresult is a complex and sophisticated infrastructure that has explodedin importance in just the past few years.

Consequently, there are challenges surrounding how individual serversfulfill growing client requests and connections from ‘the outsideworld’, and how these challenges impact organization of complex anddiscrete files, data, databases, and storage. PCI-X and Infiniband aretwo solutions that will greatly improve I/O performance, and thereforeincrease broadband access to remotely stored/cached digital information.

Infiniband is an architecture and specification for data flow betweenprocessors and I/O devices that promises greater bandwidth and almostunlimited expandability in tomorrow's computer systems. In the next fewyears, Infiniband is expected to gradually replace the existingPeripheral Component Interconnect (PCI) shared-bus approach used in mostof today's personal computers and servers. Offering throughput of up to2.5 gigabytes per second and support for up to 64,000 addressabledevices, this architecture also promises increased reliability, bettersharing of data between clustered processors, and built-in security.Infiniband is the result of merging two competing designs, Future I/O,developed by Compaq, IBM, and Hewlett-Packard, with Next Generation I/O,developed by Intel, Microsoft, and Sun Microsystems. For a short timebefore the group came up with a new name, Infiniband was called SystemI/O.

PCI-X (Peripheral Component Interconnect Extended) is a new computer bustechnology (the “data pipes” between parts of a computer) that increasesthe speed data can move within a computer from 66 MHz to 133 MHz. Thistechnology was developed jointly by IBM, HP, and Compaq, and PCI-Xdoubles the speed and amount of data exchanged between the computerprocessor and peripherals. With the current PCI design, one 64-bit busruns at 66 MHz and additional buses move 32 bits at 66 MHz or 64 bits at33 MHz. The maximum amount of data exchanged between the processor andperipherals using the current PCI design are 532 MB per second. WithPCI-X, one 64-bit bus runs at 133 MHz with the rest running at 66 MHz,allowing for a data exchange of 1.06 GB per second. PCI-X isbackwards-compatible, meaning that you can, for example, install a PCI-Xcard in a standard PCI slot but expect a decrease in speed to 33 MHz.You can also use both PCI and PCI-X cards on the same bus, but the busspeed will run at the speed of the slowest card. PCI-X is more faulttolerant than PCI. For example, PCI-X is able to reinitialize a faultycard or take it offline before computer failure occurs.

PCI-X was designed for servers to increase performance for highbandwidth devices such as Gigabit Ethernet cards, Fibre Channel, Ultra3Small Computer System Interface, and processors that are interconnectedas a cluster. Compaq, IBM, and HP submitted PCI-X to the PCI SpecialInterest Group (Special Interest Group of the Association for ComputingMachinery) in 1998. PCI SIG approved PCI-X, and it is now an openstandard that can be adapted and used by all computer developers. PCISIG controls technical support, training and compliance testing forPCI-X. IBM, Intel, Microelectronics and Mylex plan to develop chipsetsto support PCI-X. 3Com and Adaptec intend to develop PCI-X peripherals.

To accelerate PCI-X adoption by the industry, Compaq offers PCI-Xdevelopment tools at their Web site.

When remotely storing digital information the following criteria shouldbe considered: the frequency of read access, frequency of write access,size of each access request, permissible latency, permissibleavailability, desired reliability, security, etc. Some data is accessedfrequently, yet rarely changed. Other data is frequently changed andrequires low latency access. These factors should be taken into account,but are often compromised in the “one size fits all” design andoperation of conventional data storage/caching systems.

Preferably, a data storage/caching system should be designed to bescaleable so a user can purchase only the capacity needed at anyparticular time. High reliability and high availability are alsoconsiderations as data users want remote access to data, and have becomeincreasingly intolerant of lost, damaged, and unavailable data.Unfortunately, current conventional data storage/caching architecturescompromise these factors, and no single data storage/cachingarchitecture provides a cost-effective, highly reliable, highlyavailable, and dynamically scaleable solution.

Today the end-user can have high-speed access to streaming andnon-streaming data in the form of websites, electronic text documents,graphic images, or spreadsheets stored/cached remotely by purchasingtelecommunication bandwidth in the form of a T-1 or a fractional T-3line, a Digital Subscriber Line (DSL), or through their cable TVprovider using a cable modem. However, no conventional digitalinformation storage/caching system addresses the needs of the end-usersdesire for widespread, low latency access to streaming and non-streamingmulti-media data in the form of music, TV shows, movies, radiobroadcasts, web casts, etc.

Advances in fiber optic transmission technology and its declining costhave enabled upgrades in front-end network systems such as cable TVnetwork trunk and feeder systems. Traditionally, these systems haveincreased the bandwidth of a telecommunication network sufficiently toprovide each subscriber his own dedicated channel to the head-end forreceiving compressed digital video. In addition, direct broadcastsatellite technology and other emerging wireless communicationtechnologies also provide dedicated multimedia and video channelsbetween a large number of end-users and the server systems. Personalcomputers and set top boxes for the end-user are also emerging, whichenable networked multimedia applications.

The above mentioned improvements may typically improve the overallperformance of current video server systems by a factor of only two orfour times, whereas the current need in the industry requiresimprovements in the range of 100 to 1000 times to make interactivestreaming video services economically feasible.

While the end-user (client) system and the front-end network systeminfrastructure is evolving rapidly to meet the requirement ofnon-streaming and interactive multimedia services, the constraints ofcurrent server architectures continue to be expensive and impracticalfor delivering these services. Current server systems are unable toprocess the large number of data streams that are required by streamingmultimedia and video services. The current choices of servers aretypically off-the-shelf mainframe or workstation technology basedparallel computing systems. The hardware and software in both cases isoptimized for computation intensive applications and for supportingmultiple concurrent users (time-sharing) with very limited emphasis onmoving data to and from a telecommunication network interface and theInput/Output (I/O) device.

Another key to acceptable multimedia audio and video streaming is theconcept of Quality of Service (QoS). Quality of Service generally refersto a technique for managing computer system resources, such asbandwidth, by specifying user visible parameters such as messagedelivery time. Policy rules are used to describe the operation of datapacket(s) to make these guarantees. Relevant standards for QoS in theIETF (Internet Engineering Task Force) are the RSVP (ResourceReservation Protocol) and COPS (Common Open Policy Service) protocols.RSVP allows for the reservation of bandwidth in advance, while COPSallows routers and switches to obtain policy rules from a server.

A major requirement in providing Quality of Service is the ability todeliver multi-media frame data at a guaranteed uniform rate. Failure tomaintain Quality of Service may typically result in an image that isjerky or distorted.

Traditional server system architectures have not been equipped with thefunctionality necessary for providing Quality of Service on a largescale. With an increasing load on server systems to provide streamingmultimedia applications, an increased volume of user (end-clients), andthe above mentioned deficiencies in current server system technology, aneed exists to provide a server system architecture or a new datastorage/caching system with enhanced search and access capabilitieswhich will be able to address the need of low latency, high-speed accessto data.

U.S. Pat. No. 5,758,085 assigned to the International Business Machine(IBM) Corporation partially addresses the above-named problems byproviding a plurality of intelligent switches in a Storage Area Network(SAN). When the end-user (client) makes a request to receive video andmultimedia data, a request is sent to the host processor which in turnsends a request to a plurality of intelligent switches on the SAN. Theintelligent switches include a cache for storing the requested data. Thedata is relayed directly from these switches to the end-user (client)requesting the multimedia data.

While the IBM system described above provides for the storage/caching ofdata onto switches, it does not allow the individual switches tocooperate together as a distributed architecture in order to poolbandwidth together to supply the backbone network, nor does it allow forthe data to reside directly on a telecommunication network medium.Current technology allows for only a 1-2 gigabyte data stream coming outof a single peripheral device such as an array of disks, wherein atelecommunication network backbone may accommodate a 10 gigabyte orhigher data stream. Also, in the above referenced patent, the individualswitches are not capable of working together to distribute a deliveryrequest over multiple switches for load balancing and streaming of therequested data.

United States Patent Application 20010049740, filed by Karpoff,addresses many of the shortcomings of the previously referenced IBM U.S.Pat. No. 5,758,085, by describing various systems and methods fordelivering streaming data packets to a client device, over atelecommunication network in response to a request for the data packetsfrom the client device. The client request is received by a server or acontroller device that is typically located on a network switch device.If received by a server, the server sends a request to the intelligentnetwork controller device for the transfer of the requested data to theclient.

In addition to the data storage network architecture and bus problemsdiscussed above, rapid access to and intelligent searching of data isimpeded by the requirements of traditional relational databasestructures.

It has been 16 months since terrorists attacked the United States, andfederal agencies are struggling to find a way to best share informationto prevent future acts of terrorism. The key to fighting terrorism isthe real-time free flow of information between federal agencies as wellas with state and local governments. More than ever before, successfulinterdiction is dependent upon collecting, analyzing, and appropriatelysharing information that exists in different databases, transactions,and other data points. The effective use of accurate information fromdiverse sources is critical to the success in the fight againstterrorism. There is no lack of desire to share information in acooperative way, however, there is no easy, and inexpensive solution toaccomplish the sharing of data stored in traditional databasestructures.

Recently, the FBI has chosen to pursue “investigative data warehousing”as a key technology for use in the war against terrorism. Thistechnology uses data mining and analytical software to sift through vastamounts of digital information to discover patterns and relationshipsthat point to potential criminal activity. The same technology is alsowidely used in the commercial sector to track consumer activity andpredict consumer behavior.

The FBI plans to build a data warehouse that receives information frommultiple FBI databases and sources. Eventually, this warehouse mightreceive and send warehoused data to and from other law enforcement andintelligence agencies. In the war against terrorism new informationtechnology is critical to analyzing and sharing information on areal-time basis. Also, the FBI is working to focus on analyticalcapabilities far more than it has in the past. For example, the FBImight want to put in a request for information on flight schools andaccess all the reports the FBI has written on flight schools fromvarious FBI databases and then analyze them using artificialintelligence software, however, they are far from having thiscapability, which is known as enterprise data warehousing in thebusiness world.

Data warehousing and data analysis/modeling tools are used extensivelyin the commercial sector to monitor sales in stores and automaticallyorder new stock when inventories run low, monitor individual customerbuying habits and try to influence consumer buying. The FBI isconsidering applying the same analysis techniques/tools currently usedby the private sector to search vast collections of data to identifysuspicious trends. For example, analyzing data collected in various FBIdatabases and by the Immigration and Naturalization Service, the CIA andother agencies could indicate suspicious activity that now isoverlooked. Add to that data from credit card companies, airlines,banks, phone companies and other commercial entities, and actions andevents that previously seemed innocent when considered separately, beginto trigger alarms when considered in context with other activities.

Most business executives make critical decisions based on data that'sbeen cut and sliced for them by information managers. If executivescould get closer to their core business data, they would increase theirodds of making better-informed decisions. The promise ofbusiness-intelligence software is to make existing enterprise databasesaccessible through easy-to-use analytical and reporting functions.Business-intelligence software quickly and cheaply allows organizationsto extract additional value from existing data warehouses and enterprisesystems.

Business-intelligence software is nearly useless for companies that have“dirty data.” Before any quality feedback can be produced, databasesmust have consistent categories, language, and maintenance.Unfortunately, for most organizations due to mergers and acquisitions,the tendency is toward chaos. They end up trying to use incompatibledatabases to force new data into legacy information systems.Consequently, uniform data-entry protocols are lacking, or ignored,making it difficult to implement changes. For example, a data field suchas, a supplier's name can be entered any number of ways by employees.Cleanup of such dirty data can be costly and can take from a few monthsto a few years.

Although the government has a huge effort underway for implementing datamining, business intelligence, and on-line analytical processing (OLAP)of transactions stored in traditional, structured data sources,intelligence agencies are in need of unstructured textual analysis tofind patterns in unstructured data.

One company, Maya Viz, combines various elements of collaboration,knowledge management, and business intelligence to bring data into avisual form that can be manipulated and shared. This technology wasfirst deployed in military command situations. The company's componentarchitecture aims to transform relational database information piecesinto nuggets, visualize them, and then through peer-to-peer connections,allow people to share the information with anyone.

Tacit Knowledge System's software automatically discovers expertise andactivity across large organizations, and connects people andinformation. This software taps into existing content sources such asdocument repositories and e-mail archives to discover individualexpertise and activity, and then makes end-users aware of relevantcolleagues and data.

The U.S. has approximately 170,000 people working together to preventattacks on the United States. This is an incredibly complex process,using multiple information technology systems to record informationabout case research, various memos, etc. In addition, a system thatcould tap into all those multiple information repositories and figureout who is working on what, would be phenomenally valuable to makecritical connections between different agencies, departments, andanalysts.

However, current data mining, warehousing, and business intelligencetechnologies are expensive and become difficult to implement,particularly when multiple federal, state, and local agencies becomeinvolved, all of which use their own proprietary technologies and dataformats. The constraint of these current data technologies is therequirement for predetermining storage format, for example, tablestructure and upfront analysis.

There is therefore a need for a method of storing/caching, searching,accessing, querying, and performing computations on data in the form ofdata packet(s) and/or data streams continuously transmitted for the lifeof the data on a telecommunication network, and/or a microprocessor,data bus, or electronic circuit. The resulting solution needs to be costeffective, avoid traditional I/O problems, overcome the limitations oftraditional relational database structures, and avoid other problems.

FEATURES OF THE INVENTION

A general object, feature, or advantage of the present invention is theprovision of a method for storing/caching data, which overcomes theproblems found in the prior art.

A further object, feature, or advantage of the present invention is theprovision of a method to store/cache data packet(s) by allowing them tobe continuously transmitted on a telecommunications network, and/or amicroprocessor, data bus, or electronic circuit for the life of the datapacket(s).

A further object, feature, or advantage of the present invention is theprovision of a method to store/cache data stream(s) by allowing them tobe continuously transmitted on a telecommunications network, and/or amicroprocessor, data bus, or electronic circuit for the life of the datastream(s).

A further object, feature, or advantage of the present invention is theprovision of a method for data storage/caching that is highly scalablethrough additional access points.

A further object, feature, or advantage of the present invention is theprovision of a method for data storage/caching that is highly scalablethrough additional parallel connections.

A further object, feature, or advantage of the present invention is theprovision of a method for data that can be easily erased without leavinga trace.

A further object, feature, or advantage of the present invention is theprovision of a method for data storage/caching that is redundant andfault tolerant.

A further object, feature, or advantage of the present invention is theprovision of a method for data storage/caching system that interfaceswith public protocols and standards.

A further object, feature, or advantage of the present invention is theprovision of a method for data storage/caching system that interfaceswith proprietary protocols and standards.

A further object, feature, or advantage of the present invention is theprovision of a method to access data stored/cached in the form ofcontinuously transmitted data packet(s) and/or data stream(s).

A further object, feature, or advantage of the present invention is theprovision of a method to search for data stored/cached in the form ofcontinuously transmitted data packet(s) and/or data stream(s).

A further object, feature, or advantage of the present invention is theprovision of a method to query data stored/cached in the form ofcontinuously transmitted data packet(s) and/or data stream(s).

A further object, feature, or advantage of the present invention is theprovision of a method to use data stored/cached in the form ofcontinuously transmitted data packet(s) and/or data stream(s) as achaotic database.

A further object, feature, or advantage of the present invention is theprovision of a method to use data stored/cached in the form ofcontinuously transmitted data packet(s) and/or data stream(s) as astructured linear database.

A further object, feature, or advantage of the present invention is theprovision of a method for performing computations on data stored/cachedin the form of continuously transmitted data packet(s) and/or datastream(s).

One or more of these and/or other objects, features, or advantages ofthe present invention will be apparent based on the Specification andclaims that follow.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, the present inventionprovides for storing data by persistently maintaining the data on anetwork, through retransmission. The data can be associated withexpiration properties. The expiration properties can includegeographical information and/or time attributes. The data can be inpackets or streams.

Referring to previously filed U.S. Provisional Patent No. 60/349,072 toMelick, et al, “FiberSan” was used as a name for one aspect of thepresent invention, and it is sometimes referred to as “DataSpace”throughout this disclosure, and is also referred to as a storage/cachingsystem or a network when appropriate. The term “network data packet”will now be referred to as “data packet”. The present invention relatesgenerally to storing/caching, searching, accessing, querying, andperforming computations of data persistently maintained throughretransmission, directly on the telecommunication network, and/or amicroprocessor, data bus, or electronic circuit. More particularly, thepresent invention relates to systems and methods for accessing,searching, querying, and performing computations on locally or globallydistributed data which are stored/cached in the form of data packets,protocol data units (PDUs), or protocol payloads, etc., continuouslytransmitted on a telecommunication network, and/or on a microprocessor,data bus, or electronic circuit for the life of the data packet(s).

As used herein, and unless otherwise specified, the term “data” is to beconstrued broadly to refer to a data packet, protocol data unit,protocol payload, data stream or other form of data.

As used herein, and unless otherwise specified, the term “network” is tobe construed broadly to refer to a network, a telecommunication network,or other type of network. It is to be understood that data present on amicroprocessor, data bus, or electronic circuit associated with thenetwork is still present on the network, and the microprocessor, databus, or electronic circuit of the device can be considered part of thenetwork in this context.

As used herein, the term “storing” is to include “caching” and queryingis to include “searching”. These and other terms that may have differentuses or particular connotation in particular contexts are to beconstrued broadly in defining the present invention.

Data content and applications have historically resided at the core ofthe Internet. However, in an effort to improve the performance ofcontent delivery, distributed computing has evolved. Distributedcomputing is the process of moving content and applications closer theend user, in what is referred to as the “edge”. To support thistransition, enterprises are creating their information technologysystems on top of the Internet Protocol (IP), which elevates the needfor distribution and computing closer to the “edge”.

The present invention allows data packets to “reverberate” on atelecommunication network, and/or microprocessor, data bus, orelectronic circuit, remaining unresolved for delivery, on-demand for thelife of the data packet(s) and/or data stream(s). The present inventionovercomes limitations associated with existing data storage/caching,accessing and searching technologies. The present invention createsunique new revenue opportunities for carriers to better utilize theirtelecommunication network assets.

In the preferred embodiment, the present invention provides a number ofadvantages over traditional remote data storage/caching, accessing,search, querying, and computational methods. These advantages caninclude: (1) reducing overall latency to and from remote data source(s);(2) providing direct access to data; (3) providing parallel multicastingcapabilities; (4) improving multicasting efficiency in telecommunicationnetworks; (5) reducing latency by moving the data storage/caching anddata storage/caching logic to a telecommunication network, and/ormicroprocessor, data bus, or electronic circuit which allows thesynchronized communication between individual intelligent networkcontroller devices; (6) eliminating the requirement for server or diskaccess by moving data storage/caching directly onto thetelecommunication network, and/or microprocessor, data bus, orelectronic circuit; (7) improving Quality of Service for streaming andnon-streaming data as a result of reduced latency and increasedbandwidth; (8) increasing the overall throughput and/or reliability byimplementing RAID methods on the intelligent network controllerdevice(s), for example utilizing RAID to mirror the data across two ormore separate DataSpaces; (9) reducing the time required for databackups, data replication and distribution by moving the administrativefunctions of data storage/caching onto the intelligent networkcontroller device; and (10) increasing availability by caching multiplecopies of the same data, which is particularly useful for multi-media.

In the preferred embodiment, the present invention can (1) lower thecost to store/cache data; (2) lower overall latency and improving accessspeeds; (3) provide new functionality that enables a host of newapplications/services to service providers and their customers; (4)provide a “geographic tag” to improve data cache selection and datarouting; and (5) provide new data search and query capabilities.

The present invention provides systems and methods for storing/caching,accessing, querying, searching, and performing computations of data ontelecommunication network infrastructure, which includes mediums, suchas but not limited to, fiber optic cable, Category 5 wire, coaxialcable, airwaves, ground waves, vacuum, space, etc., which have beenpartitioned by the present invention's switches, which also may serve asnodes on a telecommunication network. The present invention's switchesare enabled with intelligent network controller devices which arecapable of copying and/or forwarding data (streaming or non-streaming)stored/cached in the form of data packets, PDUs, or protocol payloads,to a client device over a telecommunication network, and/ormicroprocessor, data bus, or electronic circuit, in response to arequest for data from a client device. The client device's request fordata is received by a server, or directly by the intelligent networkcontroller device. Alternatively, the server may send a request to anintelligent network controller device to control the forwarding of therequested data packet(s) stored/cached to the client.

The switches and/or intelligent network controller devices of thepresent invention are not restricted in location and can be accessedremotely by the client. They can be co-located on the same physicaldevice as the client and attached via a network, data bus, or electroniccircuit. As an example, a switch and intelligent network controllerdevice may be co-located on a card or device interconnected by PCI orUSB bus on the same computer.

In a preferred embodiment, the present invention delivers datastored/cached on a telecommunication network infrastructure to a clientdevice making a request for data over a public and/or privatetelecommunication network, such as but not limited to, a Wide AreaNetwork (WAN), a Metropolitan Area Network (MAN), a Local Area Network(LAN), a Wireless Wide Area Network (WWAN), a Wireless Local AreaNetwork (WLAN), a Personal Area Network (PAN), or a Broadcast AreaNetwork (BAN), or any combination, or sub-combination of these, or othertelecommunication networks, and/or microprocessor, data bus, orelectronic circuit.

Alternatively, in a preferred embodiment, the client device's requestfor data may also be received directly by an intelligent networkcontroller device. The present invention's intelligent networkcontroller device includes the processing capability required foridentifying, copying or transferring the data, and delivering itdirectly to the client device without involving a server system. Thedata request may be mirrored to another controller device to handle thedata processing and delivery functions. In other cases, the controllerdevice may coordinate the delivery of the requested data using one ormore other similar controller devices, in parallel.

In another embodiment of the present invention, data requests for largeamounts of data are handled by the present invention's intelligentnetwork controller devices working in parallel synchronization, todeliver data packets stored/cached on various channels of a storagesystem, or on various disparate storage systems.

In another embodiment of the present invention, a storage system of thepresent invention may be expanded beyond the geographical bounds imposedby the present invention's switches, which are used to partition atelecommunication network, and/or microprocessor, data bus, orelectronic circuit, to include a client device as one of the nodes inthe network. In this way, a client device can efficiently andcontinually broadcast updated data packets.

In another embodiment of the present invention, a the present inventionmay be used as data storage/cache on a microprocessor, or data bus, orelectronic circuit.

These embodiments of the present invention represent variousarchitectures on which data may be stored/cached in the form of datapackets, PDUs, or protocol payloads, etc., continuously transmitted on atelecommunications network, and/or a microprocessor, data bus, orelectronic circuit for the life of the data packet.

Some of the unique applications for the present invention include, butare not limited to, a chaotic database, a DNS server update mechanism, astorage/cache system for data, file systems, and meta-data, mobile datasystems, high read use/decision support systems, content management andgeographical routing, and as a support technology for stream queryingand grid computing.

Reference to the remaining portions of the specification, including thedrawings and claims, will realize other features and advantages of thepresent invention. Further features and advantages of the presentinvention, as well as the structure and operation of various embodimentsof the present invention, are described in detail below with respect tothe accompanying drawings. In the drawings, like reference numbersindicate identical or functionally similar elements. Additionally, theleft-most digit(s) of a reference number identifies the drawing in whichthe reference number first appears.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates the components of present invention's storage/cachingsystem.

FIG. 2 illustrates a block diagram describing the intelligent networkcontroller device used on the present invention's storage/cachingsystem.

FIG. 3A is a flow chart describing a general process for storing andretrieving data and meta-data from the DataSpaces shown in FIG. 4.

FIG. 4 illustrates the preferred embodiment of the present invention, inwhich a request message is routed through a server, which communicateswith an intelligent network controller device located on a DataSpacethrough a back end network to another DataSpace.

FIG. 5 illustrates another embodiment of the present invention, in whicha request message is sent directly to an intelligent network controllerdevice on a DataSpace, which communicates with another intelligentnetwork controller device located on another DataSpace through a backend network.

FIG. 6 illustrates another embodiment of the present invention, in whicha client device becomes one of the nodes in the DataSpace.

FIG. 7A illustrates the hardware and software components including thenetwork loader used in the first prototype of the present invention.

FIG. 7B illustrates the architecture of the first prototype of thepresent invention.

FIG. 8 shows screenshots of output from the DataSpace hardware andsoftware used in a first prototype of the present invention.

FIG. 9 shows screenshots of output from the DataSpace hardware andsoftware used in a first prototype of the present invention.

FIG. 10 shows screenshots of output from the DataSpace hardware andsoftware used in a first prototype of the present invention.

FIG. 11 shows screenshots of output from the DataSpace hardware andsoftware used in a first prototype of the present invention.

FIG. 12 illustrates the architecture of the second DataSpace prototype.

FIG. 13 is a flow chart describing the general process used in thesecond DataSpace prototype to retrieve data stored in the form of datapackets.

FIG. 14 is a block diagram describing the basic architecture of aDataSpace configured in the form of an electronic circuit.

FIG. 15 is a diagram that illustrates the basic components of aDataSpace RAM switch shown in FIG. 14.

FIG. 16 illustrates the architecture of the present invention configuredas a distributed compute device.

FIG. 17 illustrates an alternate architecture of the present inventionconfigured as a distributed compute device.

FIG. 18 illustrates the architecture of the present invention configuredfor math computations.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT(S)

The present invention generally relates to storage/caching systems(DataSpace). In particular, the present invention relates to methods andsystems for storing/caching, accessing, searching, querying, andperforming computations on locally or globally distributed datastored/cached in the form of data packets, PDUs, or protocol payloads,etc., continuously transmitted on a telecommunication network, and/or amicroprocessor, data bus, or electronic circuit. In a DataSpace, theterm data storage/caching does not imply a static physicalstorage/caching device, but is defined as data in the form of datapackets, PDUs, or protocol payloads, etc., continuously transmitted ontelecommunication network, and/or microprocessor, data bus, orelectronic circuit for the life of the data packet(s).

DataSpace complements two existing computing concepts, stream queryingand grid computing. DataSpace provides a new data storage/cachetechnology that increases the speed and methods of data access,searching, querying, and computing capabilities. A DataSpace differsfrom stream querying in that it adds a level of persistence to datastreams by continuously transmitting them on the telecommunicationsnetwork, and/or a microprocessor, data bus, or electronic circuit.

The following table illustrates a comparison of key metrics between atypical disk storage medium (Seagate Cheetah disc drives) and a 244 milelong DataSpace.

TABLE 1 DATASPACE DISK DRIVE TRANSFER RATE 1 Gpbs (Gigabit Ethernet) 427Mbps SEEK TIME .382 msec  5.2 msec LATENCY 2.99 msec (244 mile network)2.99 msec

DataSpace also facilitates access within the grid computing environmentby providing an alternative method for the distribution, storage andcaching of data to remote or grid interconnected computing resources andservices. In addition, DataSpace can aid in the grid computing supportand management systems by storing “directory of services”,meta-services, resource and service access and availability information.

The STREAM project at Stanford University is supported in part by theNational Science Foundation under grant IIS-0118173. This researchproject addresses applications such as network monitoring,telecommunications data management, web personalization, manufacturing,sensor networks, and others, in which data takes the form of continuousdata streams rather than finite stored data sets. Traditional databasesystems and data processing algorithms are ill-equipped to handle datastreams effectively, and many aspects of data management and processingneed to be re-examined in the presence of data streams.

The STREAM project is investigating data management and processing inthe presence of multiple, continuous, rapid, time-varying data streams.Their work addresses problems including basic theory results,algorithms, and implementing a comprehensive prototype data streammanagement system. The STREAM project, and other data streamtechnologies are relevant to the present invention, as DataSpace is infact a persistent data stream which is continuous, although morerepetitious in nature, rapidly varying, and can be created from acompilation of multiple source streams. Therefore, the results andfindings of the STREAM project and related work will be fundamentallyuseful in creating a streaming data manager for the present invention.

The data packets, PDUs, and/or protocol payloads used by the presentinvention can be defined by any non-proprietary or proprietaryprotocols. Examples of non-proprietary protocols for defining datapackets include but are not limited to, Asynchronous Transfer Mode (ATM)frames, Internet Protocol (IP) packets, or Cellular Digital Packet Data(CPCD) packets (commonly known as IP over wireless). Two proprietaryprotocols for defining data packets are Novell's IPX/SPX protocol andApple Computer's AppleTalk protocol. Closed network systems used byfinancial institutions, government, or the military might use aproprietary data packet protocol internally, particularly where securityis an issue. A unique proprietary data packet protocol, in which a datapacket's payload is defined and structured as a linear database, isdisclosed in the previously mentioned U.S. patent application Ser. No.09/698,793. Another proprietary data packet protocol, Lightwaves DataLink (LDL), was designed for use with the system described in U.S.Provisional Patent application 60/376,592, to Melick, et al, entitledHigh Number Base Encoded Ultra Wideband Over Guided Lines And Non-GuidedNarrow Band Radio, incorporated herein by reference. This proprietaryprotocol is based on Simple Data Link (SDL), which is a variable lengthATM protocol.

Referring to FIG. 1, the general topology of a DataSpace 130, 130′ usedin the preferred embodiment and several of the alternative networkembodiments of the present invention is shown. A DataSpace 130, 130′ iscomprised of three basic components: (1) switches 100, 100′ are equippedwith a intelligent network controller device 200 as defined in FIG. 2,(2) ‘working telecommunication medium’ 110 and ‘spare telecommunicationmedium’ 111 to couple DataSpace switches 100, 100′, and (3) a data link120 to connect the DataSpace 130, 130′ to a telecommunication network.

The switches 100, 100′ can be configured in many different form factors,such as but not limited to carrier class soft switches, data networkelements such as switches, routers, hubs, and bridges, programmablehardware with architecture similar to a Lucent Excel LNX programmableswitch, host controllers, etc. The functions of the switch 100, 100′ maybe reduced to a microprocessor, such as but not limited to, anApplication Specific Integrated Circuit (ASIC), or Field ProgrammableGate Array (FPGA) for deployment in a client device, such as, but notlimited to, a personal computer 401, cable TV set-top box 402, PDA 403,ATM 404, radio 405, as shown in FIG. 4, FIG. 5, and FIG. 6, whichenables the client device, a personal computer 401, cable TV set-top box402, PDA 403, ATM 404, radio 405, to be used as one of the nodes in aDataSpace 130, 130′.

The DataSpace switches 100, 100′ can also be configured to operate as anew class of telecommunication switches based on an emerging technologyknown as XML switching. XML switching deployed on switches 100, 100′ mayuse XML as a universal translator, which can automatically adapt to andtransform to a myriad of XML schemas, protocols, and standards whichhave been, and are being developed for a variety of industries andapplications. XML switching intelligence on a telecommunication networkwill enable a freer and faster flow of information from the DataSpace130 to other DataSpaces 130′, as shown in FIG. 4, FIG. 5, and FIG. 6systems, applications, and end-users.

Sarvega, Inc. manufactures Extensible Mark-up Language (XML) switchespowered by their XML Event Stream Operating System™ (XESOS™) technologyto allow the intelligent routing of XML-based data at wire speeds.

The explosive growth of XML usage in corporate IT infrastructures hasmade it necessary to address the performance issues inherent in XMLprocessing. XML processing tasks, such as XSL transformation, schemavalidation, XPATH-based classification, XML security, and intelligentrouting are all inherently processing intensive. This has placed asignificant burden on existing server infrastructure that is notoptimized to perform these tasks. Sarvega's switches have been designedto provide the specialized processing and intelligence required totransparently offload XML processing from the general purpose serverinfrastructure.

Sarvega's XML switches could be modified to allow DataSpace switch logicto reside directly on their switch in order to give them the advantagesassociated with continuously transmitted data packet(s) or data streamscontaining information within XML constructs. This combination ofSarvega's mature XML processing integrated with the present invention'sswitching and processing logic would be one method of enablingapplications such as a chaotic database.

Alternatively, Sarvega's XML switches could be incorporated as astand-alone component(s) in a DataSpace controlled by presentinvention's switching hardware and logic.

The present invention can incorporate XML switching technology to storeand tag geographic information for data packets, and be used forsubsequent routing on a DataSpace 130, 130′. Assigning and storing aDataSpace packet's information can be a self-provisioning feature,particularly for mobile devices that employ radio frequency positioningtechnology such as the NAVSTAR Global Positioning System (GPS), Loran,Shoran, or the emerging technology for precise local positioning knownas ultra wideband (UWB). Another self-provisioned method, which isunique to the present invention uses geographic information related toan end-user, or hardware device, particularly for stationary devicessuch as desktop personal computers, Automatic Teller Machines (ATM), faxmachines, etc. This end-user information can be acquired for use on theDataSpace through the use of an onboard software utility which mapsmanually input information such as zip code, address, or telephonenumber to a latitude and longitude.

In addition to the above mentioned self-provisioning methods forobtaining and incorporating geographic information for use inintelligent data packet routing, there are many other public and privatesources for obtaining and provisioning geographic information asdescribed in U.S. Provisional Patent Application Ser. No. 60/732,505 toMelick, et al., entitled UNIFED MESSAGE SYSTEM, incorporated herein byreference. The United States Postal Service databases and systems maplongitude and latitude to street address and zip code. Corporateentitles, such as telephone companies, maintain cross-referencedatabases for telephone number to latitude and longitude information,Also, fee-based service providers, such as Quova, map IP addresses tolatitude and longitude.

Geographical information obtained from of the above mentioned means canbe used to augment the intelligent routing capabilities of DataSpaceswitches 100, 100′ in order to cache data, particularly data that isgeographically/demographically sensitive, closer to an end-user.

The geographical information and/or time information can be used inassociating expiration properties with the data. Based on expiration orrelated binding properties, access to data can be limited. For example,data can be accessible only during pre-defined time frames. Geographicaland time information can be combined to create more complex bindingproperties.

A DataSpace 130, 130′ uses a telecommunication network infrastructure tostore/cache data in the form of data packets, PDUs, or protocolpayloads, etc., continuously transmitted on the network for the life ofthe data packet(s) or data stream(s). The time of transit betweenDataSpace switches 100, 100′ will be relatively short in comparison tothe time stored in the Random Access Memory (RAM) on these switches. TheRAM will contain a processing buffer for sending a data packet andsubsequent receiving and re-sending of the same data packet. This buffermay also be provided by intelligently routing data packets over aminimum length of network transmission medium to buffer the timerequired to process a data packet. On a network the length oftransmission medium can be reduced by intelligently routing data packetsover multiple channels to buffer the time between the sending andreceiving of the same data packet. Alternatively, an air gap betweenterrestrial satellite ground stations and orbiting satellites may beused to create a buffer between the sending and receiving of the samedata packet. A buffering device on a DataSpace 130, 130′ that uses fiberoptic medium could be created in the future by intentionally slowinglight down, or even stopping it completely by directing it intosuper-cooled, sodium atoms as described in a Jan. 25, 2001 article byBill Delaney, posted on the CNN.com web-site (C:\WINDOWS\TemporaryInternet Files\OLK40D4\CNN.com—US—Bill Delaney Harvard scientist stopslight—Jan. 25, 2001.html).

For matters of practicality and creating easy migration paths to thewidespread use of DataSpaces 130, 130′ in the marketplace, the DataSpaceswitches 100, 100′ are designed to receive data packets forstorage/caching over data link 120, using any standard messagingprotocol. Typical examples of messaging protocols used in accordancewith the present invention for sending messages between the variousnodes on the DataSpace 130, 130′, include but are not limited to:Internet Small Computer System Interface (iSCSI), Small Computer SystemInterface (SCSI), Fiber Channel (FC), Infiniband (IB), Gigabit Ethernet(GE), Ten Gigabit Ethernet (10GE), and Synchronous Optical Network(SONET). Any messaging protocol, or combination of messaging protocolswhich allows for the communication of messages between the various nodesof a telecommunication network are intended to be within the scope ofthe invention.

In the preferred embodiment of the present invention, the DataSpaceswitches 100, 100′ are coupled to working transmission medium 110 andspare transmission medium 111 configured in a bi-directional ringtopology. In the preferred embodiment of the present invention, theworking transmission medium 110 and spare transmission medium 111 arefiber optic cabling, but in other non-carrier class implementations, thetransmission medium could be copper or even airwaves. The sparetransmission medium 111, or what is commonly referred to as ‘protectring fibers’ in the telecommunication industry, provide redundancy andgreater reliability as a mechanism to re-route traffic in the oppositedirection in case working transmission medium 110 fails. In addition, ifa switch 100, 100′ on the working transmission medium 110 fails, theback-up switch 100, 100′ on the spare transmission medium 111 willautomatically take over.

The present invention can be deployed on a fiber optic network with amodified version of a standard protocol such as, but not limited toSONET. SONET is a standard way to multiplex high-speed traffic frommultiple vendors multiplexing onto fiber optic backbone cabling. SONETis a four layered protocol. Layer 1 is the Photonic Layer, whichconverts electronic signals into optical signals and vice versa. Layer2, the Section Layer, monitors the condition of the transmission betweenthe SONET equipment and optical amplifiers. Layer 3 is the Line Layer,which synchronizes and multiplexes multiple streams, from multiplesources, into a single data stream in a uniform format. Layer 4 is thePath Layer which assembles and disassembles voice and data into frames.In the present invention, the Path Layer could use a modified addressingconvention to identify which data packets are to be stored/cached on theDataSpace 130, 130′ by continuously transmitting them on thestorage/caching network for the life of the data packet. DenseWavelength Division Multiplexing (DWDM) is a new high-speed, replacementtechnology for SONET which can also be used as the basis to create aDataSpace 130, 130′.

By increasing the size of a telecommunication network, and/or speed ofthe network elements, such as switches, bridges and/or routers, or byadding DataSpace switches 100, 100′, a DataSpace 130, 130′ is a highlyscalable and load balanced data storage/caching solution without addingmuch complexity.

In the present invention, data integrity, recovery, and re-constructioncan be accomplished by a variety of methods in the case of transmissionor equipment failure, or maintenance.

Re-Directed Data—In the event a DataSpace switch, or other hardware, isremoved or ceases to function properly, the DataSpace switch responsiblefor sending data packet(s) to another DataSpace switch can re-route themto a back-up DataSpace switch, or another DataSpace. The process ofre-directing data packet(s) can be manual and/or automated.

RAID-type Implementation—Critical data residing on a DataSpace may beprotected using a similar method found in RAID algorithms. Separate andredundant DataSpaces can be used to implement the striping and/ormirroring of data across multiple DataSpaces. Under the proper mirroringconfiguration, if one DataSpace becomes inoperable or inaccessible,another DataSpace(s) can re-construct data and service requests asneeded.

Integrity Node—A DataSpace integrity node can be placed on the DataSpaceto check for any number of possible situations indicating corrupted ormissing data. This node could be used in conjunction with the RAID-typeImplementation for data recovery. The Integrity Node could be directedto look for: out-of-order packets (if ordering is required), and/ormissing packets by performing packet count (if the number of packets isstatic), and/or CRC checking on segments of a DataSpace.

Alternative Data Storage—A DataSpace can export data packet(s) and/ordata stream(s) to other storage devices as a safe-store method in theevent a DataSpace needs to backup to a form of permanent storage. Inaddition, if data on a DataSpace becomes lost or corrupted, a DataSpacecan re-construct the data from the safe-storage device. Examples ofthese safe-storage devices include disk drives, floppy drives, and/oroptical drives.

Referring to FIG. 2, the intelligent network controller device 200, 200′includes a Central Processing Unit (CPU) 230, data cache memory 220which is used to support processing, a systems interface 201 (typicallya Field Programmable Gate Array (FPGA) or Application SpecificIntegrated Circuit (ASIC))—which may include an optional computationengine 202, which can be used to accelerate operations such as RAIDcheck sum calculations, encryption, compression, and data routing=), andat least one communication port 270, 270′ which is used to communicatewith the external network, other intelligent network controller devices200, 200′, or other forms of data storage/caching, such as SANs, NAS,etc., or other peripherals. At a minimum, the intelligent networkcontroller device 200, 200′ would also include a communication port 260,which is used to retrieve data from a DataSpace 130, 130′. Optionally,it may include a server communication port 250 used to connect withservers, and/or a LAN communication port 240 may be included forconnection to a LAN, and/or a communication port 260′ for sending datato another network controller device 200, 200′. The intelligent networkcontroller device 200, 200′ may include additional communication portsof any of the previously described, as required. The intelligent networkcontroller device 200, 200′ is not limited to the types of communicationports previously described.

More than one of the above communication ports 240, 250, 260, 260′ 270,270′ may be combined into a single communication port. Also, each of thecommunication ports 240, 250, 260, 260′, 270, 270′ may be operating on adifferent protocol, such as but not limited to iSCSI, Infiniband, FiberChannel, SONET, etc., or proprietary protocols. It should be appreciatedto one skilled in the art that the term ‘communication port’ is intendedto be construed broadly to include any combination and sub-combinationof different communication ports for allowing the data and messagingcommunication using appropriate protocols over a telecommunicationnetwork.

The intelligent network controller device 200, 200′ is programmed to becapable of receiving and processing requests for data, such as but notlimited to, copying, mirroring, backing-up, performing RAID data checksum functions, data routing, administrative functions, encryption,compression, and forwarding data to a client, and/or another DataSpace130, 130′, and/or physical disks intended for data storage/caching.

Optionally, the storage/caching management and administrationfunctionality of server 420, as depicted in FIGS. 4 and 6, may beintegrated on the intelligent network controller device 200, 200′ suchthat a client may communicate directly with the controller device 200,200′ through a front-end network 410 without involving a server 420.

It should be noted that the intelligent network controller device 200,200′ is capable of executing and fulfilling client requests, such as butnot limited to, Hyper Text Transfer Protocol (HTTP), File TransferProtocol (FTP), or Real Time Streaming Protocol (RTSP), etc.

The DataSpace 130, 130′ hardware configuration may be modified dependingupon the optimization of the application(s) required by the client upona telecommunication network configuration. U.S. Pat. No. 6,148,414,Brown, et al, which is hereby incorporated by reference in its entiretyfor all purposes, provides useful device configurations for embodimentsof the present invention.

As the present invention stores data in the form of data packets, PDUs,or protocol payloads, etc., continuously transmitted on thestorage/caching network for the life of the data packet(s) and/or datastream(s), U.S. Patent Application 20010049740, which is herebyincorporated by reference in its entirety for all purposes, providesuseful systems and methods for delivering data content, particularlystreaming data content, to a client over a telecommunication network inresponse to a request for the data content from the client device.

In the following description, like reference numbers indicate identicalor functionally similar elements. Additionally, the left-most digit(s)of a reference number identifies the drawing in which the referencenumber first appears.

FIG. 3A, is a flow chart for the network architecture as shown in FIG.4, and illustrates the general process for retrieving data which isstored/cached in the form of data and meta-data packet(s) continuouslytransmitted on a DataSpace 130′ and 130 respectively. Metadata is “dataabout data” which allows computers to process information moreeffectively. The request for data and meta-data is mediated by server420 in response to a request by a client device 401, 402, 403, 404, 405.

FIG. 4 illustrates an embodiment of a network topology of the presentinvention in which a client device, a personal computer 401, cable TVset-top box 402, PDA 403, ATM 404, radio 405, communicates throughfront-end network 410 to server 420 to request data stored/cached onDataSpace 130 and/or 130′.

When the data packet(s) and/or data stream(s) requested by a clientdevice is not located on DataSpace 130, the server 420 communicates withthe intelligent network controller device 200′ which is located onanother DataSpace 130′. The server 420 typically forwards a notificationmessage over a back-end network 440 which is connected to the DataSpace130′. The back-end network 440 may also be connected to anotherfront-end network 410′. The telecommunication links between the networks410, 130, 440, 130′, 410′ may include optical fiber, for example, butmay be comprised of other mediums, such as but not limited to, fiberoptic cable, Category 5 wire, coaxial cable, airwaves, ground waves,vacuum, space, etc.

FIG. 5 illustrates another embodiment of a network topology the presentinvention in which a client device, a personal computer 401, cable TVset-top box 402, PDA 403, ATM 404, radio 405, communicates throughfront-end network 410 to a DataSpace switch 100, 100′ operating onDataSpace 130, 130′ to request data stored/cached on DataSpace 130and/or 130′.

When the data packet(s) and/or data stream(s) requested by a clientdevice, a personal computer 401, cable TV set-top box 402, PDA 403, ATM404, radio 405, is not located on DataSpace 130, the intelligent networkcontroller device 200 located on a DataSpace 130 communicates with aintelligent network controller device 200′ which is located on anotherDataSpace 130′. The intelligent network controller device 200 forwards anotification message over a back-end network 440 which is connected tothe DataSpace 130′. The back-end network 440 may also be connected toanother front-end network 410′. The telecommunication links between thenetworks 410, 130, 440, 130′, 410′ may include optical fiber, forexample, but may be comprised of other mediums, such as but not limitedto, fiber optic cable, Category 5 wire, coaxial cable, airwaves, groundwaves, vacuum, space, etc.

FIG. 6 illustrates an embodiment of the present invention in which aclient device 401′ serves as a broadcast or repeater node in a DataSpace130, 130′. A client device personal computer 401, cable TV set-top box402, PDA 403, ATM 404, radio 405 communicates through front-end network410 through server 420, to DataSpace 130, 130′ to request datastored/cached on DataSpace 130 and/or 130′ which is continually beingupdated by client device 401′. Client device 401′ can be equipped withDataSpace switches 100, 100′ and intelligent controller devices 200,200′ that allow it to store/cache data in the form of data packet(s),PDUs, and/or protocol packets, etc., which are transmitted back andforth between client device 401′ and DataSpace 130, 130′ for the life ofthe data packet(s), PDUs, and/or protocol packets. This embodiment ofthe present invention allows a client device 401′ to continuallybroadcast current data.

The inventors have built two DataSpace prototypes. The first is a remotestorage/caching system accessed by a client via iSCSI over a front-endnetwork to a DataSpace. A “block-like file system” architecturetypically found on traditional magnetic media was implemented on aDataSpace and integrated with an iSCSI target server on a DataSpaceintelligent controller device. An iSCSI target server routed iSCSI blockI/O operation requests from the client to the DataSpace instead of atraditional magnetic media device. The “block-like file system” on theDataSpace consists of data packets representing media blockscontinuously transmitted for the life of the data packets.

The second prototype is remote storage/cache system accessed by a clientdevice's web browser through an HTTP server integrated with a DataSpaceintelligent controller device. The web pages and graphics requested bythe client device's browser HTTP requests are retrieved from a DataSpacewhere they were previously stored/cached. The prototypes demonstratedthe ability of a network to serve as a data storage/cache mechanism bycontinuously transmitting data packets between network devices.

DATASPACE PROTOTYPE ONE—FIGS. 7 through 11 illustrate the firstDataSpace prototype. This prototype consists of two major components: aclient device, specifically a PC that initiates DataSpace clientrequests, and two DataSpace prototype switches built on a PC platform.The code for this prototype was disclosed in U.S. Provisional PatentApplications, Ser. No. 60/366,803 ENTITLED SYSTEM AND METHOD FORSTORING/CACHING DATA ON TRANSMISSION INFRASTRUCTURE, previouslyincorporated by reference.

The DataSpace prototypes include a client device, such as, but notlimited to, a personal computer, cable TV set-top box, PDA, that iscapable of establishing and maintaining a network connection to theDataSpace. Table 1A shows the data flow from an application to aprototype switch from a stack perspective:

TABLE 1A Layer Stack Operation Example Component 1 Application Forexample: streaming video, editor 2 O/S Linux 3 File System O/S or FileSystem 4 ISCSI Intel iSCSI layer 5 TCP/IP TCP/IP Network Stack 6Ethernet/Physical Ethernet

The DataSpace prototype switch receives requests for datastorage/caching operations and performs them on the DataSpace. TheDataSpace prototype may also include load balancing, data integrityoperations, and transfer of data to another data storage medium, suchas, but not limited to, a physical disk farm or another DataSpace.

Since one goal of the prototypes is to demonstrate the compatibility toco-exist with existing and emerging standards and protocols, InternetSmall Computer Systems Interface, iSCSI was selected as the keyintegration point for the first prototype. Others, such as, but notlimited to Hypertext Transfer Protocol (HTTP), File Transfer Protocol(FTP), Real Time Protocol (RTP), Network Time Protocol (NTP), PostOffice Protocol (POP3), etc., can be implemented as well since theprimary software developed is built independent of delivery method to aserver(s).

For the first prototype, Intel's iSCSI software was used for thefollowing reasons. First, iSCSI is an acknowledged standard based upontwo established technologies: SCSI and TCP/IP. Both of these technologystandards are well integrated into existing computing systems and wouldmake integration easier to accomplish with the initial prototype. SinceiSCSI encapsulates the data storage/caching operations within TCP/IPpackets, it will be simple to examine these packets upon receipt andtranslate them into DataSpace storage/caching operations.

The Intel iSCSI software was found in public source:www.sourceforge.com. Some minor modifications to the Intel source werenecessary to provide interconnection to the DataSpace prototype system.

The DataSpace client application creates file operations that result iniSCSI transactions to an iSCSI target device which uses the DataSpace asa storage/caching device.

iSCSI protocol handles requests between a client and a target. In thefirst prototype the DataSpace switch is the target. The client loads adevice driver for an iSCSI initiator. This device networks with theiSCSI target on the prototype switch. The client creates a diskpartition and file system on the iSCSI device, which is currently donetoday with traditional SCSI devices. The client mounts the iSCSI device,then runs an application, or other programs, that use the iSCSI devicedriver to send disk operation requests to the DataSpace. The DataSpaceiSCSI target routes the iSCSI requests to the DataSpace for action, andreturns the response.

The DataSpace prototypes were built around the concept of representingdisk blocks on a network in the form of continuously transmitted datapacket(s). The prototypes were designed using 512 byte data packets. Forexample, if we wanted to install a disk that has 40,000 512 byte blocks,then we would create a DataSpace that has 40,000 512 byte data packets.

The client's file system uses the iSCSI initiator to issue read/writeblock request(s) on the iSCSI target device. Upon receipt of these blockoperation requests, the iSCSI target places the request into sharedmemory that is accessed by the DataSpace switch.

While the switch is processing DataSpace packets, it scans the memorytable to determine if the DataSpace data packet currently in process hasa request from the iSCSI target. If there is a request, for example, ablock read, write, verify, etc., then 1) the operation is performed onthe DataSpace packet, 2) the memory table is updated and 3) the packetis sent to the next DataSpace prototype switch.

The following describes the basic algorithm of how the operations occuron the prototype switch:

Initialize parameters

Setup connections to prototype switches

Continuous loop:

-   -   Listen on receiving port for DataSpace data packets    -   Read and assemble DataSpace data packet    -   Check the DataSpace iSCSI target request table    -   If request posted for DataSpace data packet, then perform        request and update the iSCSI target request table    -   Send DataSpace data packet to next prototype switch

End of loop

The following is a description of DataSpace Prototype One architectureas shown in FIG. 7A. This architecture emulates a distributed andshareable RAM disk, in which a DataSpace data packet 732A is used inplace of a memory block on a computer RAM disk. Instead of read/writingdata to a hard storage device, a client device, such as, but not limitedto, a PC 401, read/writes data to data packets continuously transmittedon a network, microprocessor, data bus, or electronic circuit.

DataSpace Prototype One consists of two prototype devices 700, 750configured as PC's running Redhat Linux 7.2 operating system 712.Prototype devices 700, 750 are interconnected via standard Ethernetnetwork to an Ethernet 10/100baseT router/hub 740. The DataSpaceprototype device 1, 700 is one physical PC unit that consists of twological devices, client device 710 and server device 720. The hardwareused for DataSpace Prototype Device 1, 700 is a Dell Dimension XPS T700rpersonal computer. The hardware used for DataSpace Prototype Device 2,750 is a Dell Optiplex GXpro personal computer. The router/hub 740 is aDlink DI-704.

Intel iSCSI software is used to send and receive SCSI instructionsbetween the file system 713 and/or operating system 712 and the iSCSItarget device 721. During this operation, iSCSI initiator 714 createsinformational and error messages to log file 715.

Intel iSCSI software was modified to allow the storage/caching deviceinstructions from client 710 via iSCSI initiator 714 to server 720 to beperformed on the DataSpace storage/cache prototype. During operation ofthe iSCSI target 721, information or error logging messages are sent tovideo display device 722.

As part of the DataSpace Prototype One setup, the DataSpace data packetloader 730 creates and sends a configurable number of DataSpace datapackets 732A to the prototype switch 728 via Ethernet. For example, ifone desires to build a 10 MG DataSpace with 512 byte datastorage/caching blocks, the DataSpace packet loader 730 will send 20,000DataSpace data packets 732A to prototype switch 728.

For DataSpace Prototype One, each DataSpace data packet 732A is a TCP/IPpacket formatted for subsequent iSCSI operations. DataSpace data packet732A contains space reserved for the DataSpace header 733A, and a spacereserved for a data storage/caching segment 734A. The DataSpace header733A is for information pertaining to the operation of the DataSpace130, 130′, and also includes a field called “block ID”. In addition tocontaining a “block ID”, the DataSpace header 733A could also bemodified to include additional meta-data associated with data containedin data segment 734A and other relevant DataSpace information. TheDataSpace loader 730 inserts a unique block number into this field foreach DataSpace data packet 732A created starting at value zero (0) andincreasing by one (1). The DataSpace data storage/caching segment 734Areserves one “block-like file system” block per data packet forstoring/caching data.

DataSpace data packet 732A becomes a DataSpace data packet 732B oncedata has been written to header 733A (which becomes a 733B) and datastorage/caching segment 734A (which becomes a 734B).

After all the required DataSpace data packets 732A have been created andsent by the DataSpace loader 730 to prototype switch 728, the DataSpaceloader 730 creates and sends out a DataSpace loader control packet 731to prototype switch 728. This loader control packet 731 contains thenumber of packets created for the DataSpace by the DataSpace loader 730.Like other DataSpace data packets 732A used for the prototype, once theDataSpace loader control packet 731 is sent to prototype switch 728, itwill be continuously transmitted on the DataSpace.

DataSpace loader 730 creates and sends out DataSpace data packets 732Ato the prototype switch 728. Prototype switch 728 receives and processes(see Table 3 below) DataSpace data packets 732A and sends them onto thenext prototype switch 728′. These DataSpace data packets 732A arecontinuously transmitted on the DataSpace, and available for read/writeoperations.

The following Table depicts the steps for the operation of the DataSpacepacket loader 730.

TABLE 2 DATASPACE PACKET LOADER OPERATION STEPS 1. Startup andinitialization 2. Establish TCP/IP connection to other DataSpaceprototype switch. 3. Begin sending DataSpace data packets equal to thenumber of DataSpace data packets specified in the program startupparameters. 4. Create new DataSpace data packet and insert the blocknumber starting with value zero (0) for first block and incrementing byone (1) thereafter. 5. Insert into DataSpace packet with TCP/IP addressinformation for a DataSpace prototype switch. 6. Send new DataSpacepacket to the DataSpace prototype switch specified in the programstartup parameters. 7. Repeat process starting at Step 4 until allDataSpace packets requested in the startup parameters have been createdand sent 8. Create a DataSpace informational packet by placing a valueequal to the number of DataSpace data packets into the informationpacket. Add the TCP/IP address information for a prototype switch andthen send to the DataSpace prototype switch specified in startupparameters.

During operation, the DataSpace loader 730 provides information or errorlogging messages to video display device 722′.

All the DataSpace data packets 732A, 732B, loader control packets 731,and switch control packets 729 will be continuously transmitted on theDataSpace network from prototype switch 728 to prototype switch 728′ toprototype switch 728 to prototype switch 728′, etc. This continuous“reverberation” of DataSpace data packets 732A, 732B, 729, 731 betweenprototype switches 728 and 728′ illustrates one of the key components ofthe DataSpace concept.

The following is a detailed description of the function of prototypeswitches 728, 728′ as shown in FIG. 7B.

Prototype switch 728′ is configured as a switch running in the normalmode and performs the operations as described below in Table 3:

TABLE 3 THE STEPS FOR THE PROTOTYPE SWITCH (NORMAL MODE)  1. Startup andInitialization  2. Attach or create request and semaphore tables.  3.Establish TCP/IP connection to other DataSpace prototype switches. Onceconnected, send a control packet to next DataSpace prototype switch.  4.Begin listening and receiving DataSpace packets.  5. For each packetreceived, create a new DataSpace packet to be sent to the next DataSpaceprototype switch.  6. Scan the request table to see if a request existsfor the particular DataSpace packet received. If a request in therequest table exists, then perform the required operation as specifiedin the request's operation field. 6a. If the operation field indicatesis a READ request, then 1) copy contents in the DataSpace packetreceived into the request table and 2) copy contents of receivedDataSpace packet into new DataSpace packet. 6b. If the operation is aWRITE request, then copy contents of the request table into the newDataSpace packet. 6c. If the operation is a VERIFY request, then copycontents of DataSpace packet received into the new DataSpace packet. 6d.If the operation is an INFORMATION request, then 1) copy contents of theDataSpace packet received into the request table and 2) copy contents ofreceived DataSpace packet into new DataSpace packet. 6e. ALL requestsupon completion will have the operation field changed to indicate theoperation is complete.  7. If no request exists in the request table fora DataSpace packet, then copy contents of DataSpace packet received intothe new DataSpace packet.  8. Update new DataSpace packet with TCP/IPaddress information for the next Prototype switch.  9. Update orincrement any counters or operation parameters in the new DataSpacepacket. 10. Send new DataSpace packet to next Prototype switch. 11.Repeat process starting at step 5 until Prototype switch is terminated.

As part of a prototype switch's 728, 728′ startup operation, once aTCP/IP network connection has been established with its next prototypeswitch 728, 728′, it sends out a prototype switch control packet 729onto the DataSpace, which is continuously transmitted for the life ofthe DataSpace. For example, prototype switch 728′ sends its switchcontrol packet 729 to prototype switch 728 and prototype switch 728sends its switch control packet 729 to prototype switch 728′. During thedevelopment of the prototype, the switch control packet 729 was used fordebugging and testing purposes. It can also serve as a “pulse”indicating the status of the DataSpace 130, 130′, contains switchspecific information to be used by other DataSpace devices formanagement, performance, and analysis operations.

When an application 711 such as “vi”, a UNIX-based text editor, is usedfor editing a file that is stored/cached on a DataSpace, it results inseveral file system requests to the operating system 712 and/or filesystem 713. These requests result into iSCSI instructions that are sentfrom the iSCSI initiator 714 to the iSCSI target 721 in the logicalserver device 720. The iSCSI target 721 is the modified Intel iSCSItarget software containing added DataSpace functionality.

When iSCSI target 721 receives an iSCSI request from the iSCSI initiator714, it translates the iSCSI request into a DataSpace request(s) andplaces the request(s) as entry(ies) 725 into the DataSpace request table724. The request table 724 contains a configurable number of re-usabletable entries 725. For most of the prototype testing, 40 shared memoryentries were created for use.

The DataSpace also uses a semaphore table 726 that contains anassociated semaphore table entry 727 mapped to each DataSpace requesttable entry 725. The prototype switch 728′, iSCSI target 721, and theDataSpace software test tool have access to write, read and modifyrequest table entries 725 in the request table 724. As a result, theprototype switch 728′, iSCSI target 721, and the DataSpace software testtool perform lock and unlock operations to a specific DataSpace requesttable entry 725 by applying the appropriate semaphore operation on theassociated DataSpace semaphore table entry 727. The use of semaphores isa common compute method used to manage access to shared resources suchas memory table entries. By implementing semaphores, resources can notsimultaneously accessed by more than one competing process; thus,removing the potential for memory corruption.

The DataSpace request table 724 and the DataSpace Semaphore table 726can be created by either the prototype switch 728′, the iSCSI Target721, or the DataSpace software test tool.

The DataSpace request operations, as shown in Table 4 below, can beperformed on the DataSpace request table 725 entry depending on thereceived iSCSI request:

TABLE 4 DATASPACE OPERATIONS DEPENDING ON ISCSI REQUESTS 1. Read—Readdata from data segment of the DataSpace data packet into DataSpacerequest table entry. 2. Write—Write data from block data in DataSpacerequest table entry into associated data segment of DataSpace datapacket. 3. Verify—Verify a specific DataSpace data packet exists. 4.Information—Obtain information from DataSpace loader control data packetinto DataSpace request table entry. Used to obtain size of DataSpaceprototype for the DataSpace components and DataSpace software test tool.

During operation of the prototype switch 728′ in normal mode, it canprovide information or error logging messages to video display device722″.

During operation of the prototype switch 728 in passthru mode, it canprovide information or error logging messages to video display device722′″.

When the prototype switch 728 operating in passthru mode receives a datapacket 732A and/or 732B, it performs the operations as shown in Table 5below:

TABLE 5 PROTOTYPE SWITCH OPERATION (PASSTHRU MODE) 1. Startup andInitialization 2. Attach or create request and semaphore tables. 3.Establish TCP/IP connection to other DataSpace prototype switches. Onceconnected, send a control packet to next DataSpace prototype switch. 4.Begin listening and receiving for DataSpace packets. 5. For each packetreceived, create a new DataSpace packet to be sent to the next DataSpaceprototype switch. 6. Copy contents of DataSpace packet received into thenew DataSpace packet. 7. Update new DataSpace packet with TCP/IP addressinformation for the next Prototype switch. 8. Update or increment anycounters or operation parameters in the new DataSpace packet. 9. Sendnew DataSpace packet to next Prototype switch. 10. Repeat processstarting at step 5 until Prototype switch is terminated.

FIG. 8 is a screenshot showing output from the hardware and softwareused in the previously described prototype of the present invention.Window 801 shows activity on the DataSpace prototype as video display722′″, as shown in FIGS. 7A and 7B from DataSpace prototype device 2,750, specifically the prototype switch 728. Window 802 shows activity onthe DataSpace prototype as video display 722″, as shown in FIGS. 7A and7B, from DataSpace prototype device 1, 700, specifically the prototypeswitch 728′ component.

FIG. 9 is a screenshot showing output from the hardware and softwareused in the previously prototype of the present invention. Window 901shows activity on the DataSpace prototype as video display 722′, asshown in FIG. 7A during the creation of the disk partition and themaking of the file system 713. Window 902 shows activity on theDataSpace prototype as video display 722, as shown in FIGS. 7A and 7B,from the iSCSI Target 721.

FIG. 10 is a screenshot showing output from the hardware and softwareused in the previously described prototype of the present invention.Window 1001 shows activity on the DataSpace prototype as video display722′, as shown in FIG. 7A during the mounting of the file system 713,the synchronization (flush file system buffers) of the file system 713,the unmounting of the file system 713, and the remounting of the filesystem 713 on the DataSpace prototype. Window 1002 shows activity on theDataSpace prototype as video display 722, as shown in FIGS. 7A and 7B,from the iSCSI Target 721.

FIG. 11 is a screenshot showing output from the hardware and softwareused in the previously described prototype of the present invention.Window 1101 shows activity on the DataSpace prototype as video display722′″ and 722″ respectively, as shown in FIGS. 7A and 7B, during theset-up of prototype switches 728, 728′. Window 1102 shows activity onthe DataSpace prototype as output 722′, as shown in FIG. 7A from theDataSpace packet loader 730. Window 1103 shows informational output ofDataSpace packet loader 730 to video display 722′.

DATASPACE PROTOTYPE TWO—DataSpace Prototype Two integrates a DataSpacewith an open-source Apache Hypertext Transfer Protocol (HTTP) web serverfound at http://www.apache.org. In this DataSpace prototype, allhardware and software components were used in DataSpace Prototype Onewith the exception of the iSCSI components.

In addition to the DataSpace Prototype One components, two othersoftware programs were created. The first program provides the methodfor uploading Hypertext Markup Language (HTML) web pages, GraphicInterchange Format (GIF), and Joint Photographic Experts Group (JPEG)images/graphics to the DataSpace for storage/caching. After theDataSpace has been initialized, including the loading of data packets inthe manner outlined in the first DataSpace prototype, the HTML, GIF andJPEG files and graphics can be uploaded to the DataSpace.

The second program is a Common Gateway Interface (CGI) program that isresponsible for processing the CGI requests received by the Apache HTTPweb server from the client. The CGI program extracts the requested HTML,JPEG or GIF information stored/cached on the DataSpace by issuing arequest to the DataSpace switch. The DataSpace switch upon receipt ofthe request from the CGI program fulfills the request and returns theinformation requested (HTML, JPEG or GIF format) back to the CGIprogram. The CGI program returns the information requested (HTML, JPEGor GIF format) back to the Apache HTTP web server, which forwards theresponse back to the requesting client.

The actual Apache source code may be modified, or a specific modulewithin Apache source code or framework may be created for tighter andmore efficient integration with the DataSpace that would eliminateimplementation of a CGI program. Also, a more detailed and sophisticatedmethod may be implemented to increase server functionality by storingassociated meta information on the DataSpace, including, but not limitedto, multiple information identification indicators (filename, serialnumber, descriptor), description, key words, size, author, time ofupload, geographic tag, access rights and expiration date.

In addition, since CGI is an established method used for adding orextending HTTP server functionality, a similar prototype may be builtwith any number of web server technologies, including, but not limitedto, a small proprietary HTTP server specifically developed for use witha DataSpace, or other commercially available server like Microsoft'sInternet Information Server (IIS).

Similar levels of DataSpace and server integration may be built usingother protocols, including, but not limited to, File Transfer Protocol(FTP) for accessing/storing/caching files, Real Time Protocol (RTP) forthe transport of real-time data, including audio and video, Network TimeProtocol (NTP), Post Office Protocol Version 3 (POP3) foraccessing/storing/caching e-mail, Real Networks Protocol foraccessing/storing/caching streaming media, Apple's QuickTime Streamingserver for accessing/storing/caching streaming media, etc.

FIG. 12 illustrates the embodiment of the network topology of DataSpacePrototype Two in which a client device, a personal computer 401communicates through front-end network 410, to server 420 to requestdata stored/cached on DataSpace 130. This prototype was operatedlocally, where the front end network 410 was a LAN, and was alsooperated remotely, where the front end network 410 consisted of an ISPand the public telecommunication network(s).

FIG. 13, is a flow chart for the network architecture as shown in FIG.12, and illustrates the general process for delivering datastored/cached in the form of data packet(s) continuously transmitted ona DataSpace 130. The request for data is mediated by server 420, runningApache software, in response to a HTTP request by a client device 401.The server 420 running Apache software executes a CGI program in orderto return the requested data to the client device 401.

FIG. 14 illustrates a DataSpace configured as an electronic circuit1400. Electronic circuit 1400 is comprised of three basic components:(1) DataSpace Random Access Memory switches (RAM switches) 1420, 1420′as defined in FIG. 15; (2) transmission medium 1410, 1410′; and (3) datalinks 1430, 1430′ which connect the DataSpace RAM switches 1420, 1420′to a compute device, telecommunication network, etc.

Electronic circuit 1400 stores/caches data in a similar manner describedin FIGS. 4, 5, and 6, with the exception that data is stored/cached inthe form of continuously transmitted data packet(s) and/or data streamswithout the use of a telecommunication network to interconnect theswitches on a DataSpace.

The electronic circuit 1400 can be contained in many different formfactors, such as, but not limited to carrier class soft switches, datanetwork elements such as switches, routers, hubs, and bridges,programmable hardware with architecture similar to a Lucent Excel LNXprogrammable switch, host controllers, and client devices such as, butnot limited to PLCs, and PCs 401, TVs 402, PDAs, 403, ATMs 404, radios405, as shown in FIGS. 4, 5, and 6.

The electronic circuit 1400 may be configured to operate using anemerging technology known as XML switching/routing. XMLswitching/routing is explained in more detail below in the applicationssection of the present invention.

Electronic circuit 1400 can be designed to use a protocol, such as, butnot limited to, Small Computer System Interface (SCSI), to encapsulatedata packets for storage/caching as previously discussed in the presentinvention.

Alternatively, electronic circuit 1400 can employ structured datastreams to store/cache data. A structured data steam has been describedin U.S. patent application Ser. No. 09/698,793 entitled METHOD OFTRANSMITTING DATA INCLUDING A STRUCTURED LINEAR DATABASE to Melick, etal, previously incorporated.

In this embodiment of the present invention, the DataSpace RAM switches1420, 1420′ are coupled via transmission medium 1410, 1410′. A singlepair of transmission mediums 1410, 1410′ is shown, but the electroniccircuit is highly scalable in a parallel manner and the DataSpace RAMswitches 1420, 1420′ may include additional ports to reverberate dataover additional transmission mediums. A uni-directional flow betweenDataSpace RAM switches 1420 and 1420′ is shown, but may be configured asbi-directional data flow.

The electronic circuit 1400 can have additional DataSpace RAM switches1420, 1420′ added to balance the load, or increase the number of nodesavailable for tapping into the data flowing on the circuit.

Data integrity on the electronic circuit 1400 can be accomplished byimplementing RAID-type methods in which data packet(s), or data streams,are striped, check summed and mirrored on the same electronic circuit1400, or across multiple electronic circuits 1400 operating in concertin order to enable data integrity and rapid data reconstruction in thecase of a transmission failure.

The DataSpace RAM switches 1420, 1420′ communicate and transmit datawith other electronic circuitry, or microprocessors through data links1430, 1430′. A single data link 1430, 1430′ is shown for each DataSpaceRAM switch 1420, 1420′ respectively, but additional ports may beincluded in order to provide additional parallel data links.

The electronic circuit 1400 may be configured to operate as a photoniccircuit, or as a circuit using pulse-based methods, similar to thosedescribed in U.S. Provisional Patent application 60/376,592, to Melick,et al, entitled High Number Base Encoded Ultra Wideband Over GuidedLines And Non-Guided Narrow Band Radio, previously incorporated byreference.

FIG. 15 illustrates the basic architecture of DataSpace RAM switch 1420,1420′. The DataSpace RAM switches 1420 or 1420′ include a CentralProcessing Unit (CPU) 230, a data memory cache 220 which is used tosupport processing, a systems interface 201, which may be configured ina device such as, but not limited to, a Field Programmable Gate Array(FPGA) or Application Specific Integrated Circuit (ASIC). The Systemsinterface 201 may include an optional computation engine 202, which canbe used to accelerate computational operations, and at least one pair oftransmission mediums 1410, 1410′ coupled to another DataSpace RAM switch1420 or 1420′. A DataSpace RAM switch 1420, 1420′ may include additionalparallel transmission mediums 1410″, 1410′″ to another DataSpace RAMswitch 1420, 1420′ to increase the throughput speed of the electroniccircuit 1400 as described in FIG. 14. Data links 1430, 1430′ connectDataSpace RAM switches 1420, 1420′ to a compute device,telecommunication network, etc.

The DataSpace RAM switch 1420, 1420′ is programmed to be capable ofreceiving and processing requests for data in the form of data packet(s)and/or data streams, such as but not limited to, copying, mirroring,backing-up, performing RAID data check sum functions, data routing,administrative functions, encryption, compression, and forwarding datato another DataSpace RAM switch 1420, 1420′, or other RAM, or physicaldisks intended for data storage/caching.

DATASPACE APPLICATIONS—Applications of DataSpace technology can beapplied to both wireline and/or wireless networks, and include, but arenot limited to, a chaotic database, Domain Name Service (DNS), astorage/cache system for data, file systems, and meta-data, mobile datasystems, high read use/decision support systems, content management andgeographical routing, and as a support technology for stream queryingand grid computing.

CACHING APPLICATION—Caching allows for more efficient use ofhigh-latency physical storage mediums such as magnetic disks. Byimplementing a cache, time-costly disk I/O operations are minimizedresulting in reduced access times and contention for disk resources.When data, such as file directory information is continually accessed ona typical storage device like magnetic media, it makes sense to storethat information in a low-latency memory location such as cache. Usingcache, once the first read to a disk is performed the cache is alsoupdated with the content retrieved by the disk read request. As aresult, subsequent read requests for the same data element previouslyretrieved will be served from the cache in lieu of performing anotherdisk operation.

Cache can be co-located locally with the file system and/or operatingsystem, or remotely on a storage system such as a Storage Area Network(SAN), traditional networked server, and storage cluster. In the case ofremote storage devices, the cache can be located on the storage systemand is delivered over a standard network transport, such as, but notlimited to TCP/IP. Using the present invention to cache data on aDataSpace allows subsequent requests for data to a remote networkedstorage/caching device without having to traverse the entire network.

In addition, multiple DataSpaces can be implemented and configured toprovide distributed caching.

CONTENT MANAGEMENT AND GEOGRAPHICAL ROUTING APPLICATION—Data that ishighly accessed for Internet web sites and multi-media can be cached tomany DataSpaces. For example, a DVD movie or series of movies can bestored/cached on a network closer to the edge or cached on otherDataSpaces between the originating source and the user, and continuouslytransmitted indefinitely for quick access by users.

Combined with geographical routing information, data can be tailored fora user's demographics. A DataSpace can maintain information about usersserved via geographical routing information and intelligently cache datain relevant DataSpace(s) closest to the user.

FILE SYSTEMS APPLICATION—Traditionally, file systems typically storerelevant information on the data storage/caching device they aremanaging. Sometimes this data is stored/cached in “superblocks”, orblocks of data on a disk device. When file system information isstored/cached on a DataSpace instead of the actual disk, managing filesystems would be more easily distributed, and many file systemoperations would become more efficient and faster.

MOBILE DATA SYSTEMS APPLICATION—When multiple entities or assets are ina mobile environment, such as, but not limited to people, vehicles,freight, luggage, military assets, etc., each of these can be nodeswithin a hybrid DataSpace. A hybrid DataSpace includes both wireless andwireline network transport mediums. Among the benefits of this are, (1)high-speed access to real-time data by DataSpace nodes; (2) faster datadistribution to multiple DataSpaces; and (3) ease in adding additionalDataSpace monitoring or intelligence nodes.

DECISION SUPPORT SYSTEMS APPLICATION—When data is characterized as highread/low write, traditional data systems require a central data store(s)to be updated before current data is available to users. Examplesinclude, asset tracking, monitoring systems, telematics, and broadcastdata. Also, central data store(s) may provide other access points via adistribution, or replication model, to other data storage/cachingsystems and/or devices.

A DataSpace provides immediate access to data for a large number ofusers without the costs normally incurred by adding access for manyusers to traditional data systems. A DataSpace is suited to thisenvironment because the processing latency at a DataSpace node isnegligible in comparison to the contention of many users vying fortraditional storage/caching resources. When additional DataSpaceresources are required to support additional users, the addition ofDataSpace nodes allows for faster distribution of data rather than thetraditional processing of data via a sophisticated distribution and/orreplication environment that requires data to be completely distributedand/or replicated before it can become available to all users.

SEARCH ENGINE APPLICATION—The DataSpace may be configured for use as anInternet search engine. When a company, user, content provider, etc.,wishes to provide searching capabilities for their data, such as but notlimited to, keywords, languages, phrases, geographical location, timestamps, speech patterns, recording, video, graphics, meta informationand other associated data elements, etc., it can be uploaded and storedon a DataSpace(s). DataSpace switches can be programmed to use variousmethods to search data using criteria, such as but not limited to,geographic region, languages, expiration of search information,keywords, meta information, patterns, categories, etc., and to controlaccess and permission restrictions.

Current Internet search engines, such as but not limited to, Yahoo!,Metacrawler, Google, etc., could redirect search requests to aDataSpace, use the DataSpace to search and/or utilize a DataSpace as aresource for their own “bot” or information acquisition technologies.Users could subscribe to a DataSpace and quickly upload, change, add,delete, or modify search criteria and information stored on a DataSpace.When service providers use a DataSpace to update or “push” data theyhave more control on subsequent DataSpace search requests. For example,a web-site owner distributing information regarding their web-site to aDataSpace may want search results to be determined by time-of-day,day-of-week, special of the day, inventory counts, closeouts, pricing orother parameters. In addition, the web site owner may want to providedifferent results for different users including, but not limited to, thecontent, the language, and/or dialect based upon the origin of theDataSpace search request. A common example of time-of-day routing can befound in the telephony market where specific 800 numbers are routedbased upon parameters such as time-of-day and/or originating calllocation. In these cases, many 800 number customers are allowed accessto telephone systems that implement their routing decisions based on amyriad of reasons. Similarly, a DataSpace allows users to updatespecific information real-time on a DataSpace that would yield differentsearch engine results.

If a service provider stores the actual web content on the DataSpace, aDataSpace could generate and return results based upon the contentstored. For example, a web page stored on a DataSpace that contains anIowa basketball sports story titled “Iowa Women Beat Purdue Men 90-3.”In addition, the web content may contain other information keywords ormeta-tags for search engine purposes, such as, “purdue, iowa, scores,sports, college, basketball”, etc. When the DataSpace is searching for arequest, it will evaluate all the information stored within a page, suchas the actual score, title, or news story content itself in combinationwith, and/or information pertaining to searching stored in meta tags aspreviously described.

A DataSpace-based search engine could provide the ability to expire orrestrict access to searching. This could allow for different levels ofsearch capability for both the user and content provider. Examplesinclude: (1) A news story with a related MP3 file has an expiration datefor searching because the news story is no longer relevant; (2)versioning of content for different users; (3) access restrictions byISP, IP address (range or specific), users; (4) geographic location,grouping, company, service etc.; (5) service level restrictions bycontent provider, user, DataSpace provider, DataSpace subscriber, etc.

Likewise, hosting providers or content providers, such as AOL,Microsoft, WorldCom, UUNet, Akamai, etc., could upload information aboutcontent, and store, service, manage, and create data directly on theDataSpace. This could be done automatically and dynamically at any timeinterval, or as changes are made to the content.

DOMAIN NAME SERVICES APPLICATION—Another application for the presentinvention is using a DataSpace as a mechanism to update and maintainDomain Name Services (DNS), or to be a DNS server itself. DNSinformation on a DataSpace could provide for instantaneous updates forDNS servers located as an example, in Botswana, the United States, andChina simultaneously. Today, when an application or device needs toresolve a domain name, it typically goes to a DNS server, which is notupdated in real-time.

A DataSpace may have global reach, and could be a centralizeddistribution system for DNS services. Registrars could update aDataSpace in near real-time, and ISP providers can use the DataSpace forup-to-date DNS resolution directly, or indirectly, for their customers.One benefit to using the present invention for real-time DNS services ishaving more dynamic capability for re-routing requests by time-of-day,load, geography, outages, type of request, etc.

LINEAR DATABASE APPLICATION—A complimentary technology to a DataSpace isLinear Databases. Linear Databases are described in U.S. patentapplication Ser. No. 09/698,793 entitled METHOD OF TRANSMITTING DATAINCLUDING A STRUCTURED LINEAR DATABASE to Melick, et al. A LinearDatabase defines a structure for data stored in the form of a datapacket(s), and can be used as a universal data interchange method.

GRID COMPUTING APPLICATION—Grid computing is the application of manydistributed computer resources in a network dedicated to a singleproblem at the same time. Among the cornerstones of grid computing arethe cataloging and management of compute and network resources, and toprovide access to these resources using application programminginterfaces (APIs) and tools.

An example is collecting data, such as space radio signals at manycollection points, and distributing the data to various computefacilities for processing. Depending on the task, compute resource(s)located on the grid may be needed for pure raw processing power orbecause it contains specific hardware and/or software developed tocomplete a specific task.

In addition, sets of data may be of interest to many different partieslocated on the gird for their own analysis and tasks. An example of thistype of data would be weather data that would be useful to agriculture,aviation, defense and other private and public entities. This use ofdata can require many types of regulated/unregulated distribution dataaccess methods. If an entity located on a grid wishes to use this kindof data, they either must pull it from a data store or have it pushed ordelivered to them.

Grid computing is a methodology and architecture intended to create asystem that easily integrates and automates some of the nuances specificto tasks that require high-contention devices, e.g. super-computerresources for sophisticated modeling. Super-computing resources aredistributed to a few global locations relative to other types ofavailable processing resources and typically require some form ofscheduling and permission prior to acquiring the resource. The gridallows for sizeable tasks to be broken down into manageable sub-tasksthat could be spread across multiple resources. By parsing thesesub-tasks among different resources, one could reduce the total amountof work typically required of a super-computer oriented task and/orbreak a task up among different super-computer resources.

Depending on the application, sizes of the datasets used in gridcomputing can be quite large and range in size greater than 10 millionobjects representing gigabytes and terabytes of data. When dealing withdifferent data use methods, distribution of resources and types ofdatasets, a DataSpace can aid in the processing of this data by cachingand storing it on the network so that more than one compute resource canaccess it simultaneously. This would allow for the use of multipleresources for analyzing data interconnected by a DataSpace.

A DataSpace is not a replacement for current data storage methods,however it can provide persistence of data objects, which can be usefulin a grid system configured for performing tasks across multiple systemsrelating to the same object. An example of object processing would be aseries of interconnected systems providing services for travelers. Thepassenger “object” would be passed among different systems performingdifferent functions, for example, booking, ticketing, baggage, security,billing, etc.

A DataSpace is also useful in the event that an object is cached for useby different systems. When tasks are serialized, once an object has beenprocessed and placed on the DataSpace, the next available resource canperform the next task for the object without having to employ asophisticated queuing scheme.

As referenced in other areas of the DataSpace and chaotic databasesapplication, an object database could easily be developed for storingobjects of different sizes and formats. In addition, a DataSpaceprovides a mechanism to store “object meta” information, such asprocessing history, processing performance, operation and measurementinformation, origin, access, etc.

A DataSpace can be useful for caching data used by many systems withinthe same timeframe. An example of this type of data would be thereal-time monitoring and analysis by many different military systems forbattlefield information. Using a DataSpace to store/cache the real-timebattlefield data eliminates the need to send the data to a common storefor analysis systems to pull from.

The predominant benefit a DataSpace provides to grid computing is theability to add persistence to objects and data elements to make themreadily available at the time a resource on the grid becomes available.A DataSpace can span different geographic locations and can be easilyintegrated into a grid. Within the context of grid computing, aDataSpace can be viewed as a large shareable RAM drive that can beeasily accessed by all the interconnected grid devices.

CHAOS DATABASE APPLICATION—One of the problems with using traditionalelement storage systems such as a relational database is the need todefine database elements such as, tables and indexes prior to theimport, creation or storage of data elements. While advances intechnologies such as XML and predictive data modeling are aiding in theeffort to reduce the amount of pre-implementation design and planningrequired for the storage of data, they are all constrained by theunderlying data access speed limits posed by magnetic media devices.

Another limit of traditional database systems is the requirement for athorough understanding of the data to be received or stored. While insome instances this is a very easy task, in other situations it can bevery difficult and challenging. There are two trends impacting datastorage in the future. The first trend is that more data from moredistinct and unknown organizations will need to be processed. Secondly,realizing that some data has a short lifetime of usefulness, a pushtowards analyzing data closer to real-time will increase the demand forminimizing the amount of resources and time spent performing traditionalpre-implementation database activities.

A DataSpace may be configured to provide a new type of element storagesystem we call Chaos Element Base (CEB). The CEB is focused primarilytowards an element storage system that supports intense queries,volatile storage elements with limited updates, the import of storageelements in their original format with little modification from manysources and formats, and the use of meta-data information related to thestorage elements.

CEBs equip the end-user with the ability to perform their own queriesand reporting thus reducing dependency on the IT department. Querying aCEB will employ a natural-type of access similar to those found ontoday's Internet search engines. Other advanced searching, querying andreporting applications may be layered on a CEB providing additionalfunctionality. As an example, a SQL-type of query engine may be islayered on the CEB to provide end-users with an applicable SQLinstruction set.

Another example of a type of querying application that can beimplemented on a chaotic database is associated with the Stanford'sStream Data Manager (STREAM) systems being development at StanfordUniversity. While STREAM systems will focus on continuous data feeds,the CEB focuses on providing a persistent data stream. Queryingtechnologies developed for STREAM such as OpenCQ and NiagraraCQ, supportcontinuous queries on data streams, and can be applied to a CEB.

Another area a CEB can be implemented is for the storage/caching andsharing of business objects. This is accomplished without converting anobject into a traditional database thus reducing the complexity ofintegration. As a CEB stores data independent of formats, the storage ofobjects of different size and complexity are minimized in comparison toforce-fitting data into traditional database technologies.

When data objects are frequently updated and shared with multiplesystems it is essential to provide a method of locking to eliminate therisk of updating the same record simultaneously. There are several typesof locking procedures that would be applicable to a CEB, includingexclusive locks and read-only locks. These methods of locking can beimplemented on a chaotic database using the “meta” area of data elementsin the CEB architecture, which allows only one client access to aparticular data at any given point, eliminating the risk of contentionamong clients.

In some cases, an object has a short lifespan and yet needs to be sharedamong disparate systems. A CEB can provide this level of functionalityas well and also provide object expiration rules for object garbagecollection typically found in object-oriented systems today.

When data is stored in its original format, it provides two advantages.The original context in which the data was received is preserved, whichprovides added meaning to the data, and the steps for importing dataelements into a CEB are eliminated, or substantially reduced vs. atraditional relational database. The steps eliminated or reducedinclude: (1) pre-definition of data elements; (2) design of relationaldatabase SQL table and indexes; (3) creation of SQL table; and (4) theimport of data and insertion into SQL table(s).

Data may be imported directly into a CEB, thus reducing the time,resources, and analysis typically required to load data into atraditional relational database. This is important when dealing withdata that is real-time, time-sensitive and/or has a short-shelf life.

DATA WITHOUT A TRACE—In some applications it may be beneficial to deletewithout leaving a trace. When deleting information from a generalcomputing device's traditional storage device, such as magneticmedia-based disk drives, its existence is removed from the eyes of anoperating system, file system and user, however, the fingerprint orphysical properties of the data remain on the device. An example of adata trace includes pits created in magnetic media devices, such as diskdrives. Unless additional safe-guards are implemented, the data willremain in its original physical form on the storage unit until it isoverwritten by another operation.

Today, special programs provide random generation of data to “wipe” afile from a disk. This makes it difficult, if not impossible, toreconstruct the data element once it has been wiped. These programs areslow since it requires numerous I/O operations to be performed on thedisk as described in Norton's Wipe Info utility describing U.S.government wiping methods.

This utility combines several wiping and overwriting processesconforming to DoD (Department of Defense) document 5220-22-M, and theNational Industrial Security Program Operating Manual, for the ultimatesecurity level when eliminating data from digital media.

Overwrites the data with 00s

Verifies the 00 overwrite

Overwrites with FFs

Verifies the FF overwrite

Writes a random value, or a value that you choose from 00 to FF

Verifies the random overwrite

Re-verifies the random overwrite to ensure that it was written correctly

Repeats as many times as you specify, up to 100

In situations where data needs to be quickly removed and its traceseliminated, a DataSpace provides this functionality. As a DataSpacestores/caches data on a telecommunication network's infrastructure,and/or an electronic circuit, data bus, or microprocessor s RAM, alltraces of data are eliminated once a DataSpace is shutdown, or the lifeof data packet(s) and/or data stream(s) has expired.

A DataSpace requires less time than traditional disk-wiping methods,because it eliminates excessive physical disk I/O's associated withtraditional medias devices such as magnetic disk drives. In addition,when a DataSpace is shutdown, it can be configured in a manner wherethere is no remaining physical fingerprint or evidence of data packet(s)and/or data stream(s) previous existence.

COMPUTE APPLICATIONS—A DataSpace's unique architecture and use ofcontinuously transmitted data packet(s) and/or data stream(s) can alsobe configured for various compute applications.

DISTRIBUTED COMPUTE DEVICE—An example of the architecture of adistributed compute device is illustrated in FIG. 16. In this embodimentof the present invention the data bus, such as, but not limited toUniversal Serial Bus (USB), or Peripheral Computer Interface (PCI)encapsulated in data packet(s) is continuously transmitted betweencomponents of a compute device connected to a DataSpace. The computedevice components include keyboard 1602, video display 722, and computerprocessor 1600, and are connected to DataSpace 130 via data link 120,120′, and 120″ respectively. The keyboard 1602 is used to input commandsinto processor 1600. The video display 722 is used to view output fromprocessor 1600. A peripheral device, a web cam 1601, is shown connectedto the DataSpace 130 via data link 120′″. In this embodiment of thepresent invention the DataSpace 130 serves as the memory and processorcache required by the processor 1600, and also serves as the data busbetween the video display 722 and processor 1600.

In this embodiment of the present invention, the general flow is shownin Table 6.

TABLE 6 FLOW FOR DATASPACE ADAPTED AS DISTRIBUTED COMPUTE DEVICE 1) Webcam 1601 generates a digital video signal. 2) Digital video signal isencapsulated in a data packet(s) and/or data stream(s). 3) Datapacket(s) and/or data stream(s) are continuously transmitted on aDataSpace 130. 4) Processor 1600 processes the data packet(s) and/ordata stream(s) being received from web cam 1601 and outputs processedvideo signal data onto DataSpace 130. 5) Display 722 receives andignores raw output from web cam 1601. 6) Display 722 receives anddisplays processed output from processor 1600.

As an example, the bus may be encapsulated in Lightwaves Data Link(LDL). LDL is a proprietary protocol based on Simple Data Link (SDL),which is a variable length Asynchronous Transfer Mode (ATM) protocol.This protocol adapted for use in a DataSpace is discussed in detaillater in the present invention. The DataSpace Version octet in the LDLprivate area of the protocol can be used in a DataSpace distributedcompute device to indicate the manner data contained within the LDLpayload is to be processed, and if it is destined for a particularcompute component connected to a DataSpace 130.

The following Table illustrates the layout for a variable lengthDataSpace LDL protocol bus frame.

TABLE 7 VARIABLE DATASPACE LDL BUS FRAME LDL Header LDL Private Area LDLPayload Payload LDL LDL LDL Check Data Length Private DataSpaceDataSpace Payload Private & Length CRC Length Version Frame ID Bus AreaPayload CRC 2 octets 2 octets 1 octet 1 octet 4 octets <=65,535 octets 4octets

Alternatively, the bus frame described in Table 7 can be a fixed length.

Another example of distributed compute device architecture isillustrated in FIG. 17. In this embodiment of the present invention thefunctionality of the computer processor 1600 shown in FIG. 16 isreplaced with a “DataSpace-within-a-DataSpace”, or what will be referredto as global processor 1700. In this embodiment, the operating system(O/S) and file system are continuously transmitted on DataSpace 130, asshown with a dashed line, are encapsulated in continuously transmitteddata packet(s) and/or data stream(s). The processor cache, ROM forprocessor software, and data storage are continuously transmitted onDataSpace 130.

DataSpace switches 100, 100′, 100″ are described in FIGS. 1 and 2. Thevideo display 722 is connected to DataSpace 130 via data link 120′, andis used to view output processed by the global processor 1700. Thekeyboard 1602 is connected to DataSpace 130 via data link 120, and isused to input commands into processors 1701, 1701′, and 1701″. Aperipheral device, a web cam 1601, is shown connected to the DataSpace130 via data link 120′″.

The O/S and file system continuously transmitted on DataSpace 130perform traditional task management procedures such as, but not limitedto, task scheduling, file management, and resource and servicesmanagement for the processors 1701, 1701′, 1701″ connected to DataSpace130 and DataSpace 130′. In addition, DataSpace 130′reverberates datapacket(s) and/or data stream(s) containing information such as, but notlimited to, dynamic and static O/S configuration and operationalinformation, semaphores, device drivers, users, system status, andinstruction sets.

DataSpace 130′ reverberates data packet(s) and/or data stream(s)containing information such as, but not limited to, processor cache,shared memory, and instruction sets, etc., for use by processors 1701,1701′, and 1701″. (Delete One Of The Periods)

TRANSMETA-LIKE COMPUTE DEVICE—Transmeta Corporation manufactures theCrusoe processor, which is an x86-compatible chip specially designed forthe handheld and lightweight mobile computing market. The Crusoeprocessor is a hardware-software hybrid chip that consumes significantlyless power than standard processors by transferring the function ofmillions of transistors with their proprietary “code-morphing” software.

The Code Morphing software is designed to dynamically translate x86instructions into VLIW (Very Long Instruction Word) instructions for theunderlying Crusoe hardware engine. The Code Morphing software resides inflash ROM. The Code Morphing software consists of two main modules thatwork in conjunction to implement the functions of an x86 processor.

The Interpreter software contains a module that interprets x86instructions one at a time, much like a traditional microprocessor. TheInterpreter functionality also filters infrequently executed code frombeing needlessly optimized and gathers run time statistical informationabout the x86 instructions it sees for determining whether optimizationsare necessary.

The Translator software detects critical, frequently used x86instruction sequences. The Code Morphing software invokes a module thatrecompiles the x86 instructions into optimized VLIW instructions, called“Translations.” The native translations reduce the number ofinstructions executed and results in better performance.

Further efficiencies are possible by saving the translations in memorythat is inaccessible to normal x86 code. This special memory area isnamed the “Translation Cache” and allows the Code Morphing software tore-use translations and eliminate redundancies. Upon encounteringpreviously translated x86 instruction sequences, the Code Morphingsoftware skips the translation process and executes the cachedtranslation directly out of the Translation Cache.

Caching and re-using translations exploits the high degree of repetitiontypically found in real world workloads. The Code Morphing softwarematches repeated executions with entries in the Translation Cache andthe optimized translation is executed at full speed with minimaloverhead. The initial cost of the translation is amortized over repeatedexecutions.

A DataSpace 130, 130′ as shown in FIGS. 16 and 17 can be adapted for usewith Transmeta, or Transmeta-like processors to enhance computefunctionality on a DataSpace 130, 130′. The DataSpace 130, 130′ canfunction as the ROM normally used to store the Code Morphing software,including interpreter and translator modules, and also the translatorcache. The DataSpace 130, 130′ would encapsulate, and continuouslytransmit the Code Morphing software and translator cache in datapacket(s) and/or data stream(s).

It is not the goal of the present invention to adapt a DataSpace for usewith Transmeta, and Transmeta-like processors in order to increase thebattery life of mobile devices, but to convert a DataSpace for use acompute device that is easily scalable. Another benefit of a DataSpaceintegrated with a Transmeta, or Transmeta-like processor would be tomodify, upgrade and change Transmeta, or Transmeta-like processors' ROMbased on the task the Transmeta processor is to perform.

MATH COMPUTATION DEVICE—One example of a DataSpace configured for mathcomputations, particularly numbers exceeding 64 bits in length, orhighly re-iterative calculations, is illustrated in FIG. 18. Data link120 is used to connect to the DataSpace 130, 130′. DataSpace switches100 ^(N), where “N” represents a “specific bit position value of anumber”, are configured to perform normal DataSpace switch 100 ^(N)functions on continuously transmitted data packet(s) and/or datastream(s), such as copying, forwarding, reading, deleting, etc. Inaddition DataSpace switches 100 ^(N) are programmed to perform additionand subtraction operations on the values represented in the “bitframes”. Where “N” equals “1” in a bit frame, DataSpace switch 100 ¹would only process bit position 1 within a bit frame and DataSpaceswitch 100 ¹²⁸ would only process bits contained within bit position“128”, etc.

After the DataSpace switch 100 ^(N) has added and subtracted bit valuesin two or more data packet(s) and/or data streams, it outputs aresultant data packet(s) and/or data stream(s) corresponding to bitposition it is processing. It may output two data packet(s) and/or twodata streams(s), and/or two values in a data stream if there iscarryover from the math processing on the bit values. The carryover datapacket(s) and/or data stream value(s) would be identified with a bitposition of N+1. As an example, DataSpace switch 100 ⁶⁴, processing bitposition 64, will output a resultant data packet, and/or data streamvalue identified as the resultant of operations performed on bits with abit position value of 64. If there is carryover, DataSpace switch 100 ⁶⁴will also output a data packet, and/or data stream value with a bitposition value of 65, which is N+1.

Table 8 illustrates an example of a bit frame designed to support mathcomputations on a DataSpare(s).

TABLE 8 VARIABLE LENGTH DATASPACE BIT FRAME FOR MATH COMPUTATIONSINTEGER NUMBER FRAME OR BIT BIT BIT BIT ID TYPE DECIMAL POSITIONOPERATION SIGN VALUE Variable 1 octet 1 octet Variable 1 octet 1 octet1octet >=1 octet

The “Number ID” value is used to indicate which operand the bit valuehas come from. The “frame type” is used to indicate what iterative valuethis is of the bit value contained in the data packet, or if the bitvalue is a resultant, etc. The “Integer or Decimal” value indicateswhich counting convention to use to determine the correct “bit position”value. The “bit position” value indicates which position the “bit value”occupied in a text string number from right-to-left for an integer, orleft-to-right for a decimal. The “bit operation” value indicates whetherthe bits are to be added or subtracted to the operand with a “Number ID”of the current data packet plus “1”. The “bit sign” value indicateswhether operands should be added or subtracted from one other. The “bitvalue” is the actual bit that is being processed.

In the architecture shown in FIG. 18, DataSpace switch 100 ¹ alsofunctions as the head-end switch which delimits the incoming text stringnumbers, and encapsulates them in a bit frame, and reverberates them onthe DataSpace 130, 130′. In addition, the DataSpace switch 100 ¹ iscapable of acting as a gate-keeper in conjunction with other DataSpacesconfigured architecturally as “AND”, “NOT”, “OR”, “NAND”, “NOR”, and“XOR” gates, etc., in order to perform any logical operation associatedwith a compute device.

Table 9 illustrates a general flow for using a DataSpace adapted formath computations.

TABLE 9 FLOW FOR DATASPACE ADAPTED FOR MATH COMPUTATIONS 1) DataSpaceswitch adapted for math computations receives numbers represented as atext strings. 2) A head-end DataSpace switch adapted for mathcomputations delimits the text string number, encapsulates it, andidentifies it by bit position and which specific operand the bit valueis associated with. 3) Delimited data packet(s) and/or data stream(s)are continuously transmitted on a DataSpace 130, 130′. 4) Theappropriate DataSpace switch for each bit position adds or subtracts thebit values represented in the data packet(s) and/or data stream(s) andencapsulates the resultant into data packet(s) and/or data stream(s) andidentifies the bit position processed including any carry-over. 5) TheDataSpace recirculates the resultant data packet(s) and/or datastream(s) until there are no carryover data packet(s) and/or data streamvalues. 6) A DataSpace switch at the head-end adapted for mathcomputations receives the resultant data packet(s) and recirculatesthem, or forwards them to another DataSpace130, 130′ if furtherprocessing is required, or returns them to the originator of thecomputation request.

In an alternative embodiment, a pair of DataSpace switches with “N”number of ports could be programmed such that the “bit positions” of atext string number were related to a port number on the DataSpaceswitches. In this embodiment, each port on a DataSpace switch would beconnected to a separate processor.

The method described above is one example of performing computations onnumbers stored/cached on a DataSpace, where long numbers could easily berepresented as polynomials. In addition, matrices for use in matrix mathcan be created using data packet(s) and/or data stream(s) indexed to aswitch port to represent rows, and columns created using indexed datapacket(s) and/or data stream(s).

LIGHTWAVES DATA LINK ADAPTED FOR DATASPACE—DataSpace is the storage ofelements such as data, video, audio, graphics, etc. on atelecommunication network. While the concept of a DataSpace is nottransport or protocol specific, the following describes a DataSpaceadapted for use with the Lightwaves Data Link (LDL) proprietaryprotocol. LDL was designed for use with the broadband transport systemdescribed in U.S. Provisional Patent, Ser. No. 60,376,592, to Melick, etal, entitled HIGH NUMBER BASE ENCODED ULTRA WIDEBAND OVER GUIDED LINESAND NON-GUIDED NARROW BAND RADIO which was previously incorporated byreference.

LDL is a proprietary protocol based on Simple Data Link (SDL), which isa variable length Asynchronous Transfer Mode (ATM) protocol.

This embodiment of the present invention includes two LDL data framearchitectures for use with a DataSpace. The first architecture uses LDLto create a DataSpace of fixed length blocks that looks like a standarddisk media device to protocols and subsystems such as iSCSI. The secondarchitecture uses LDL to create a DataSpace of variable length blocks,and is oriented towards supporting applications such as the previouslydiscussed chaotic databases.

FIXED BLOCK LENGTH LDL ADAPTED FOR DATASPACE—Table 10 illustrates thebasic structure of a fixed block length data frame:

TABLE 10 DATASPACE LDL DATA FRAME FIXED BLOCK ARCHITECTURE LDL HeaderLDL Private Area LDL Payload LDL Check LDL Payload Payload DataSpaceSector Block Data Private & Data Length Length & Payload CRC Block IDCRC 2 octets 2 octets 4 octets <=65,535 octets 4 octets

Most traditional storage systems utilize fixed-length block sizes, forexample, 256, 512, 1024 bytes. As a DataSpace is created the appropriatenumber of fixed length blocks will be created to be utilized by theconnected operating system (OS) and/or file system.

In this embodiment of the present invention, the DataSpace is configuredto contain a group of LDL fixed length frames representing data storageblocks, similar to what has been previously described in the firstprototype as illustrated in FIGS. 7 through 11. The DataSpace switchreceives block requests from an iSCSI client through an iSCSI interface,and in turn performs read and write actions on the Data space's LDLpayloads representing the memory blocks.

FIG. 4 illustrates one architecture of a DataSpace that is suitable forthe storage/caching of traditional element storage blocks. The clientdevice requests a file or element that traverses through the operatingsystem (OS) and/or file system, such as, but not limited to an iSCSItransport. The transmission over iSCSI send and receives “block read andwrite requests” to an iSCSI server device.

VARIABLE BLOCK LENGTH LDL ADAPTED FOR DATASPACE—The DataSpace LDL frameuses some of the same elements in SDL. The core DataSpace LDL data framefields are outlined in Table 11.

TABLE 11 CORE VARIABLE DATASPACE LDL DATA FRAME LDL Header LDL PrivateArea LDL Payload Payload LDL LDL Payload LDL Check Data Length PrivateDataSpace DataSpace DataSpace Data Private & Length CRC Length VersionFrame ID Area Payload CRC 2 octets 2 octets 1 octet 1 octet 4 octets<=65,535 octets 4 octets

By changing the version of the LDL data frame, a different private areacan be created in both size and content. The underlying core LDLprocessing is capable of sending and receiving data frames of varyingtypes. The ability to process the data frame is independent of thedevice's ability to end-process the frame by a higher layer application.

Table 12 illustrates some possible data frame version values anddescriptions.

TABLE 12 DATASPACE LDL VERSION TABLE LDL DataSpace Version Private LDLOctet Area Version Value Description Size Core Variable Frame 1 DefaultDataSpace LDL frame.  6 octets DataSpace DOE 12 DataSpace Directory ofElements  6 octets DataSpace DES 13 DataSpace Element Storage  6 octetsDataSpace DESE 14 DataSpace Element Storage 10 octets Extended DataSpaceMDESE 15 DataSpace Element Meta 10 octets

A DataSpace is based upon the continuous reverberation of data frames,which could be implemented with or without a directory of elements (DOE)area, which is analogous to a file system allocation tables. The primarybenefit to implementing a DOE within a DataSpace is to improve DataSpaceefficiency

The LDL Payload Data Area for a DataSpace DOE frame contains the indexto other elements within the DataSpace. The DOE information for eachelement can be defined or enhanced by using an XML-based technology.

Table 13 is an example of implementing a DataSpace DOE with pseudo-XML:

TABLE 13 DATASPACE LDL DOE WITH PSEUDO-XML <?xml version=“1.0”?><dataspace version=“1.0”>  <doe version=“1.0”>   <element>   <name>Element Name</name>    <expiry>    . . .     </expiry>   <access>     <owner>      . . .     </owner>     <digital rights>     . . .      </digital rights>    </access>    <location dspace=“X”ldlframe=“Y” offset=“Z”/>   </element>  </doe> </dataspace>

When a DOE is implemented on a DataSpace, separate LDL frames will becreated as required, for providing the DOE information to the DataSpaceswitches. Table 14 illustrates the basic LDL DOE data frame structure.

TABLE 14 DATASPACE LDL DOE DATA FRAME LDL Header LDL Private Area LDLPayload Payload LDL LDL Payload LDL Check Data Length Private DataSpaceDataSpace DataSpace Private & Length CRC Length Version Frame ID DataArea Payload CRC 2 octets 2 octets 1 octet 1 octet 4 octets <=65,535 4octets octets Variable Variable 6 12 Variable Variable Variable

The LDL payload data area for a DataSpace DOE frame contains the indexto other elements within the DataSpace. The DOE information for eachelement can be defined, for example, using an XML-based technology.

LDL frames can be allocated for storing/caching elements eitherregistered or not registered under the DataSpace DOE. The LDL frame forDataSpace Element Storage (DES) can be allocated or de-allocated asrequired, by the DataSpace management system and can contain more thanone element. Table 15 illustrates the basic LDL DES data framearchitecture.

TABLE 15 DATASPACE LDL DES DATA FRAME LDL Header LDL Private Area LDLPayload Payload LDL DataSpace LDL Payload LDL Check Data Length PrivateDataSpace Frame DataSpace Private & Length CRC Length Version ID DataArea Payload CRC 2 octets 2 octets 1 octet 1 octet 4 octets <=65,535 4octets octets Variable Variable 6 13 Variable Variable Variable

Table 16 is an illustration of the layout DataSpace LDL DES data frame.

TABLE 16 DATASPACE LDL DES EXTENDED DATA FRAME LDL Header LDL PrivateArea LDL Payload Payload LDL Next LDL Payload LDL Check Data LengthPrivate DataSpace DataSpace DataSpace DataSpace Private & Length CRCLength Version Frame ID Frame ID Data Area Payload CRC 2 octets 2 octets1 octet 1 octet 4 octets 4 octets <=65,535 4 octets octets VariableVariable 10 14 Variable Zero or Variable Variable Variable

Table 17 is an illustration of the layout of a DataSpace LDL DESextended data frame.

TABLE 17 DATASPACE LDL DES EXTENDED DATA FRAME (META) LDL Header LDLPrivate Area LDL Payload Payload LDL Next LDL Payload LDL Check DataLength Private DataSpace DataSpace DataSpace DataSpace Private & LengthCRC Length Version Frame ID Frame ID Data Area Payload CRC 2 octets 2octets 1 octet 1 octet 4 octets 4 octets <=65,535 4 octets octetsVariable Variable 10 15 Variable Zero or Variable Variable Variable

The LDL DataSpace payload area contains the data to be stored on theDataSpace. These data elements may or may not be registered with theDataSpace DOE. They are not limited to one DES data frame and could spanmultiple data frames.

When a new element is added to a DataSpace, the DataSpace switch insertsthe data element(s) into the DataSpace creating a new frame(s). In aneffort to reduce the complexity of free space management, new dataframes are allocated for new data elements as they are entered in theDataSpace.

The DataSpace will inspect the DataSpace DES and DOE areas for data thathas expired, or marked for deletion. These data packets will simply beremoved from the DataSpace.

Using LDL as a DataSpace protocol eliminates the need to manipulate thedata payloads typically required by other protocols. Other protocolsrequire this manipulation to remove any data segments within the datapayloads that are identical to the ones used by the protocol forsegregating the separate data payloads. As an example, if a DataSpace isimplemented in a protocol such as Ethernet, a combination of characters,such as Ethernet's 7 octet sized preamble, would be used as amarker/indicator in between multiple PDUs as a means to delineate themultiple PDUs. In this example, insertion of an element into theDataSpace DES would require substituting modified sequences that differfrom the PDU delineation sequence, with sequences identical to thoseused by the DataSpace element delineation. In addition, when data isremoved from the DataSpace DES payload for inspection or delivery, thepayload would have to be inspected again and any substituted sequencesremoved and the original sequences reconstructed. Using the LDL protocolremoves the requirement for manipulating the DES payloads on insertionand/or reading.

In cases where payloads to be stored are greater than one frame, alinkage is maintained between the frames so that the entire payload canbe extracted. The LDL DES extended frame contains a field named “NextLDL Frame ID.”, which is the next sequential 64K octets of the dataelement stored/cached on a DataSpace. This is an iterative process untilno more frames are required for storing the entire data element. Whenthe last frame containing the element is reached, its “Next LDL FrameID” field is set to a value of zero.

For some data elements, it is useful to add an additionalmeta-information frame that describes the nature of the data elementcontained in the next frame(s). Examples of this type of information aresource origination, method of delivery, etc.

When an element is inserted into a DataSpace, a separate meta-data framecan be constructed and linked to the next series of frame(s) containingthe actual data element. If the data element is managed by the DataSpaceDOE, a meta-data frame will be the first frame in the data elementstream of frames.

The fields and block sizes described in Tables 10 through 17 areexamples, and may be changed, deleted, modified, or eliminated, asrequired during implementation.

While the DataSpace has been illustrated and prototyped in a form thatwould indicate its architecture is static, a DataSpace's configurationcan also be dynamic in nature for any period of time including thelifespan of a DataSpace. DataSpace components including but not limitedto DataSpace switches, intelligent controllers, clients, networktransport mediums, processors, etc., can be inserted, removed andre-configured as required. Examples of a when DataSpace re-configurationmay be required are when DataSpace component(s) fail, DataSpacecomponents are added or removed for performance, reliability andscalability reasons, adding new functionality or processingcapabilities, new applications, and for general operation, managementand maintenance functionality.

While the present invention has been described by way of example, interms of two prototype systems, and in terms of the specificembodiments, it is to be understood that the present invention is notlimited to the disclosed embodiments. It is intended to cover variousmodifications and similar arrangements, methodologies, and relateddevices, and systems as would be apparent to those skilled in the art.Those skilled in the art to which the present invention pertains willrecognize and be able to practice additional variations in the methodsand systems described which fall within the teachings of this invention.Accordingly, all such modifications and additions are deemed to bewithin the scope of the invention.

What is claimed is:
 1. A system for persistently maintaining data usinga network for data packets, comprising: a transmission medium associatedwith the network; a plurality of switches operatively connected to thetransmission medium, each switch having an intelligent networkcontroller for delivering the data packets to a device operativelyconnected to the intelligent network controller in response to a requestfor the data packets from the device and for determining if the datapackets are unexpired based on an expiration property of the datapackets and re-transmitting unexpired data packets over the network;wherein the intelligent network controller of each of the plurality ofswitches being configured to route block input/output (I/O) operationrequests from the device to the data packets on the network; wherein theexpiration property of the data packets specifies an amount of time thedata packets are persistently maintained on the network sufficient toallow the data packets to be stored, accessed, and searched whilereverberating on the system; wherein the data packets encode a digitalvideo signal.
 2. The system of claim 1 wherein the transmission mediumis a guided medium.
 3. The system of claim 2 wherein the re-transmittingthe unexpired data packets is performed using radio frequency over theguided medium.
 4. The system of claim 3 wherein the radio frequency isultra wideband radio frequency.
 5. The system of claim 1 wherein thetransmission medium is a non-guided medium.
 6. The system of claim 5wherein the re-transmitting the unexpired data packets is performedusing radio frequency over the non-guided medium.
 7. The system of claim6 wherein the radio frequency is ultra wideband radio frequency.
 8. Asystem for persistently maintaining data using a network for datastreams, comprising: a transmission medium associated with the network;a plurality of switches operatively connected to the transmissionmedium, each switch having an intelligent network controller fordelivering at least one of the data streams to a device operativelyconnected to the intelligent network controller in response to a requestfor the at least one of the data streams from the device and fordetermining if the data streams are unexpired based on an expirationproperty of the data streams and re-transmitting unexpired data streamsover the network; wherein each of the plurality of switches beingconfigured to buffer the data packets for a time period longer than timeof transit between switches; wherein the expiration property of the datastream is specified by a XML tag and specifies an amount of time thedata streams are persistently maintained on the network sufficient toallow the data stream to be stored, accessed, and searched whilereverberating on the system; and wherein the intelligent networkcontroller of each of the plurality of switches being configured toroute block input/output (I/O) operation requests from the device to thedata streams on the network; wherein the data streams encode digitalvideo signals.
 9. The system of claim 8 wherein the transmission mediumis a guided medium.
 10. The system of claim 9 wherein there-transmitting the unexpired data packets is performed using radiofrequency over the guided medium.
 11. The system of claim 10 wherein theradio frequency is ultra wideband radio frequency.
 12. The system ofclaim 8 wherein the transmission medium is a non-guided medium.
 13. Thesystem of claim 12 wherein the re-transmitting the unexpired datapackets is performed using radio frequency over the non-guided medium.14. The system of claim 13 wherein the radio frequency is ultra widebandradio frequency.